2023-02-15 06:53:46,754 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-15 06:53:46,982 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef 2023-02-15 06:53:46,997 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-15 06:53:46,998 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35, deleteOnExit=true 2023-02-15 06:53:46,998 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-15 06:53:46,998 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/test.cache.data in system properties and HBase conf 2023-02-15 06:53:46,999 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/hadoop.tmp.dir in system properties and HBase conf 2023-02-15 06:53:46,999 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/hadoop.log.dir in system properties and HBase conf 2023-02-15 06:53:46,999 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-15 06:53:47,000 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-15 06:53:47,000 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-15 06:53:47,108 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-15 06:53:47,494 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-15 06:53:47,498 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-15 06:53:47,499 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-15 06:53:47,499 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-15 06:53:47,499 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-15 06:53:47,500 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-15 06:53:47,500 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-15 06:53:47,500 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-15 06:53:47,500 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-15 06:53:47,501 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-15 06:53:47,501 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/nfs.dump.dir in system properties and HBase conf 2023-02-15 06:53:47,502 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/java.io.tmpdir in system properties and HBase conf 2023-02-15 06:53:47,502 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-15 06:53:47,503 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-15 06:53:47,503 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-15 06:53:48,034 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-15 06:53:48,039 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-15 06:53:48,340 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-15 06:53:48,527 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-15 06:53:48,547 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-15 06:53:48,589 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-15 06:53:48,628 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/java.io.tmpdir/Jetty_localhost_35713_hdfs____.rs3nmn/webapp 2023-02-15 06:53:48,812 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35713 2023-02-15 06:53:48,833 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-15 06:53:48,833 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-15 06:53:49,380 WARN [Listener at localhost/41175] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-15 06:53:49,442 WARN [Listener at localhost/41175] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-15 06:53:49,459 WARN [Listener at localhost/41175] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-15 06:53:49,466 INFO [Listener at localhost/41175] log.Slf4jLog(67): jetty-6.1.26 2023-02-15 06:53:49,471 INFO [Listener at localhost/41175] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/java.io.tmpdir/Jetty_localhost_41207_datanode____kle22r/webapp 2023-02-15 06:53:49,572 INFO [Listener at localhost/41175] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41207 2023-02-15 06:53:49,914 WARN [Listener at localhost/37817] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-15 06:53:49,928 WARN [Listener at localhost/37817] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-15 06:53:49,932 WARN [Listener at localhost/37817] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-15 06:53:49,934 INFO [Listener at localhost/37817] log.Slf4jLog(67): jetty-6.1.26 2023-02-15 06:53:49,941 INFO [Listener at localhost/37817] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/java.io.tmpdir/Jetty_localhost_35899_datanode____t2u36l/webapp 2023-02-15 06:53:50,050 INFO [Listener at localhost/37817] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35899 2023-02-15 06:53:50,074 WARN [Listener at localhost/40191] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-15 06:53:50,087 WARN [Listener at localhost/40191] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-15 06:53:50,092 WARN [Listener at localhost/40191] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-15 06:53:50,093 INFO [Listener at localhost/40191] log.Slf4jLog(67): jetty-6.1.26 2023-02-15 06:53:50,097 INFO [Listener at localhost/40191] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/java.io.tmpdir/Jetty_localhost_40237_datanode____3ycy69/webapp 2023-02-15 06:53:50,211 INFO [Listener at localhost/40191] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40237 2023-02-15 06:53:50,222 WARN [Listener at localhost/37669] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-15 06:53:50,429 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x65cb77bb640dac58: Processing first storage report for DS-a8cfc2f8-a466-4675-bf87-0687f8b45116 from datanode 36321d6c-46ca-442e-92ed-6bcb61ed715a 2023-02-15 06:53:50,430 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x65cb77bb640dac58: from storage DS-a8cfc2f8-a466-4675-bf87-0687f8b45116 node DatanodeRegistration(127.0.0.1:46725, datanodeUuid=36321d6c-46ca-442e-92ed-6bcb61ed715a, infoPort=34463, infoSecurePort=0, ipcPort=37669, storageInfo=lv=-57;cid=testClusterID;nsid=1715849582;c=1676444028112), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-15 06:53:50,430 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb63bce49634cf760: Processing first storage report for DS-7f6910ca-5199-4aec-9cda-9050c12d81d5 from datanode 7246ed63-ed04-41ac-9719-168705d617e6 2023-02-15 06:53:50,431 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb63bce49634cf760: from storage DS-7f6910ca-5199-4aec-9cda-9050c12d81d5 node DatanodeRegistration(127.0.0.1:33311, datanodeUuid=7246ed63-ed04-41ac-9719-168705d617e6, infoPort=40041, infoSecurePort=0, ipcPort=37817, storageInfo=lv=-57;cid=testClusterID;nsid=1715849582;c=1676444028112), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-15 06:53:50,431 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2d8c4fa3a17bcb08: Processing first storage report for DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36 from datanode f7dac7ef-ce9e-4233-96d7-09bfa9824e44 2023-02-15 06:53:50,431 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2d8c4fa3a17bcb08: from storage DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36 node DatanodeRegistration(127.0.0.1:38215, datanodeUuid=f7dac7ef-ce9e-4233-96d7-09bfa9824e44, infoPort=45531, infoSecurePort=0, ipcPort=40191, storageInfo=lv=-57;cid=testClusterID;nsid=1715849582;c=1676444028112), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-15 06:53:50,431 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x65cb77bb640dac58: Processing first storage report for DS-a5756fc6-c059-4520-9305-94ee2653af16 from datanode 36321d6c-46ca-442e-92ed-6bcb61ed715a 2023-02-15 06:53:50,431 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x65cb77bb640dac58: from storage DS-a5756fc6-c059-4520-9305-94ee2653af16 node DatanodeRegistration(127.0.0.1:46725, datanodeUuid=36321d6c-46ca-442e-92ed-6bcb61ed715a, infoPort=34463, infoSecurePort=0, ipcPort=37669, storageInfo=lv=-57;cid=testClusterID;nsid=1715849582;c=1676444028112), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-15 06:53:50,431 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb63bce49634cf760: Processing first storage report for DS-a904d462-8418-4065-b0ee-f45105ed3f17 from datanode 7246ed63-ed04-41ac-9719-168705d617e6 2023-02-15 06:53:50,432 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb63bce49634cf760: from storage DS-a904d462-8418-4065-b0ee-f45105ed3f17 node DatanodeRegistration(127.0.0.1:33311, datanodeUuid=7246ed63-ed04-41ac-9719-168705d617e6, infoPort=40041, infoSecurePort=0, ipcPort=37817, storageInfo=lv=-57;cid=testClusterID;nsid=1715849582;c=1676444028112), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-15 06:53:50,432 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x2d8c4fa3a17bcb08: Processing first storage report for DS-17b7516c-8130-4648-aca5-b18683bffce2 from datanode f7dac7ef-ce9e-4233-96d7-09bfa9824e44 2023-02-15 06:53:50,432 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x2d8c4fa3a17bcb08: from storage DS-17b7516c-8130-4648-aca5-b18683bffce2 node DatanodeRegistration(127.0.0.1:38215, datanodeUuid=f7dac7ef-ce9e-4233-96d7-09bfa9824e44, infoPort=45531, infoSecurePort=0, ipcPort=40191, storageInfo=lv=-57;cid=testClusterID;nsid=1715849582;c=1676444028112), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-15 06:53:50,620 DEBUG [Listener at localhost/37669] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef 2023-02-15 06:53:50,691 INFO [Listener at localhost/37669] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/zookeeper_0, clientPort=51507, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-15 06:53:50,709 INFO [Listener at localhost/37669] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51507 2023-02-15 06:53:50,721 INFO [Listener at localhost/37669] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:50,724 INFO [Listener at localhost/37669] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:51,411 INFO [Listener at localhost/37669] util.FSUtils(479): Created version file at hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2 with version=8 2023-02-15 06:53:51,411 INFO [Listener at localhost/37669] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/hbase-staging 2023-02-15 06:53:51,742 INFO [Listener at localhost/37669] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-15 06:53:52,194 INFO [Listener at localhost/37669] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-15 06:53:52,225 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,225 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,226 INFO [Listener at localhost/37669] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-15 06:53:52,226 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,226 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-15 06:53:52,362 INFO [Listener at localhost/37669] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-15 06:53:52,430 DEBUG [Listener at localhost/37669] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-15 06:53:52,526 INFO [Listener at localhost/37669] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46669 2023-02-15 06:53:52,536 INFO [Listener at localhost/37669] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:52,538 INFO [Listener at localhost/37669] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:52,558 INFO [Listener at localhost/37669] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46669 connecting to ZooKeeper ensemble=127.0.0.1:51507 2023-02-15 06:53:52,596 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:466690x0, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-15 06:53:52,599 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:46669-0x10056c1c0920000 connected 2023-02-15 06:53:52,624 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-15 06:53:52,625 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-15 06:53:52,628 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-15 06:53:52,635 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46669 2023-02-15 06:53:52,635 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46669 2023-02-15 06:53:52,635 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46669 2023-02-15 06:53:52,636 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46669 2023-02-15 06:53:52,636 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46669 2023-02-15 06:53:52,642 INFO [Listener at localhost/37669] master.HMaster(444): hbase.rootdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2, hbase.cluster.distributed=false 2023-02-15 06:53:52,717 INFO [Listener at localhost/37669] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-15 06:53:52,717 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,717 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,717 INFO [Listener at localhost/37669] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-15 06:53:52,718 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,718 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-15 06:53:52,724 INFO [Listener at localhost/37669] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-15 06:53:52,728 INFO [Listener at localhost/37669] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35089 2023-02-15 06:53:52,730 INFO [Listener at localhost/37669] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-15 06:53:52,738 DEBUG [Listener at localhost/37669] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-15 06:53:52,739 INFO [Listener at localhost/37669] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:52,742 INFO [Listener at localhost/37669] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:52,743 INFO [Listener at localhost/37669] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35089 connecting to ZooKeeper ensemble=127.0.0.1:51507 2023-02-15 06:53:52,748 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:350890x0, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-15 06:53:52,749 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-15 06:53:52,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:35089-0x10056c1c0920001 connected 2023-02-15 06:53:52,751 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-15 06:53:52,752 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-15 06:53:52,753 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35089 2023-02-15 06:53:52,753 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35089 2023-02-15 06:53:52,754 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35089 2023-02-15 06:53:52,754 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35089 2023-02-15 06:53:52,755 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35089 2023-02-15 06:53:52,772 INFO [Listener at localhost/37669] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-15 06:53:52,772 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,773 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,773 INFO [Listener at localhost/37669] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-15 06:53:52,774 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,774 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-15 06:53:52,774 INFO [Listener at localhost/37669] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-15 06:53:52,777 INFO [Listener at localhost/37669] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40475 2023-02-15 06:53:52,778 INFO [Listener at localhost/37669] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-15 06:53:52,780 DEBUG [Listener at localhost/37669] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-15 06:53:52,782 INFO [Listener at localhost/37669] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:52,785 INFO [Listener at localhost/37669] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:52,788 INFO [Listener at localhost/37669] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40475 connecting to ZooKeeper ensemble=127.0.0.1:51507 2023-02-15 06:53:52,795 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:404750x0, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-15 06:53:52,796 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:40475-0x10056c1c0920002 connected 2023-02-15 06:53:52,796 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-15 06:53:52,797 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-15 06:53:52,798 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-15 06:53:52,799 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40475 2023-02-15 06:53:52,800 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40475 2023-02-15 06:53:52,805 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40475 2023-02-15 06:53:52,805 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40475 2023-02-15 06:53:52,806 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40475 2023-02-15 06:53:52,819 INFO [Listener at localhost/37669] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-15 06:53:52,819 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,819 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,819 INFO [Listener at localhost/37669] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-15 06:53:52,819 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-15 06:53:52,819 INFO [Listener at localhost/37669] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-15 06:53:52,820 INFO [Listener at localhost/37669] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-15 06:53:52,821 INFO [Listener at localhost/37669] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41759 2023-02-15 06:53:52,821 INFO [Listener at localhost/37669] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-15 06:53:52,825 DEBUG [Listener at localhost/37669] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-15 06:53:52,826 INFO [Listener at localhost/37669] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:52,827 INFO [Listener at localhost/37669] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:52,828 INFO [Listener at localhost/37669] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41759 connecting to ZooKeeper ensemble=127.0.0.1:51507 2023-02-15 06:53:52,831 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:417590x0, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-15 06:53:52,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41759-0x10056c1c0920003 connected 2023-02-15 06:53:52,833 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-15 06:53:52,833 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-15 06:53:52,834 DEBUG [Listener at localhost/37669] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-15 06:53:52,835 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41759 2023-02-15 06:53:52,835 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41759 2023-02-15 06:53:52,836 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41759 2023-02-15 06:53:52,840 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41759 2023-02-15 06:53:52,840 DEBUG [Listener at localhost/37669] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41759 2023-02-15 06:53:52,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,46669,1676444031550 2023-02-15 06:53:52,852 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-15 06:53:52,854 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,46669,1676444031550 2023-02-15 06:53:52,871 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-15 06:53:52,871 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-15 06:53:52,871 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-15 06:53:52,871 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-15 06:53:52,872 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:53:52,873 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-15 06:53:52,875 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-15 06:53:52,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,46669,1676444031550 from backup master directory 2023-02-15 06:53:52,879 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,46669,1676444031550 2023-02-15 06:53:52,879 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-15 06:53:52,879 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-15 06:53:52,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,46669,1676444031550 2023-02-15 06:53:52,882 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-15 06:53:52,883 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-15 06:53:52,980 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/hbase.id with ID: e60c3a26-6484-40bb-86cd-5e8a1f308e92 2023-02-15 06:53:53,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-15 06:53:53,035 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:53:53,084 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4d68b895 to 127.0.0.1:51507 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-15 06:53:53,119 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1ea0868f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-15 06:53:53,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-15 06:53:53,143 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-15 06:53:53,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-15 06:53:53,163 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-15 06:53:53,165 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-02-15 06:53:53,171 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-02-15 06:53:53,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-15 06:53:53,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/data/master/store-tmp 2023-02-15 06:53:53,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:53:53,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-15 06:53:53,268 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-15 06:53:53,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-15 06:53:53,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-15 06:53:53,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-15 06:53:53,268 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-15 06:53:53,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-02-15 06:53:53,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/WALs/jenkins-hbase4.apache.org,46669,1676444031550 2023-02-15 06:53:53,290 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46669%2C1676444031550, suffix=, logDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/WALs/jenkins-hbase4.apache.org,46669,1676444031550, archiveDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/oldWALs, maxLogs=10 2023-02-15 06:53:53,347 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46725,DS-a8cfc2f8-a466-4675-bf87-0687f8b45116,DISK] 2023-02-15 06:53:53,347 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38215,DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36,DISK] 2023-02-15 06:53:53,347 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33311,DS-7f6910ca-5199-4aec-9cda-9050c12d81d5,DISK] 2023-02-15 06:53:53,355 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-15 06:53:53,422 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/WALs/jenkins-hbase4.apache.org,46669,1676444031550/jenkins-hbase4.apache.org%2C46669%2C1676444031550.1676444033300 2023-02-15 06:53:53,423 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38215,DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36,DISK], DatanodeInfoWithStorage[127.0.0.1:33311,DS-7f6910ca-5199-4aec-9cda-9050c12d81d5,DISK], DatanodeInfoWithStorage[127.0.0.1:46725,DS-a8cfc2f8-a466-4675-bf87-0687f8b45116,DISK]] 2023-02-15 06:53:53,423 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:53:53,424 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:53:53,427 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-15 06:53:53,428 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-15 06:53:53,482 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-15 06:53:53,489 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-15 06:53:53,514 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-15 06:53:53,526 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:53,531 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-15 06:53:53,533 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-15 06:53:53,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-15 06:53:53,549 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:53:53,550 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70101318, jitterRate=0.044591039419174194}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-15 06:53:53,551 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-02-15 06:53:53,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-15 06:53:53,570 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:53:53,570 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:53:53,573 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:53:53,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:53:53,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-02-15 06:53:53,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-15 06:53:53,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-15 06:53:53,634 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-15 06:53:53,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-15 06:53:53,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-15 06:53:53,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-15 06:53:53,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-15 06:53:53,670 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-15 06:53:53,673 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:53:53,675 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-15 06:53:53,675 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-15 06:53:53,687 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-15 06:53:53,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-15 06:53:53,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-15 06:53:53,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-15 06:53:53,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-15 06:53:53,693 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:53:53,693 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,46669,1676444031550, sessionid=0x10056c1c0920000, setting cluster-up flag (Was=false) 2023-02-15 06:53:53,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:53:53,715 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-15 06:53:53,716 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46669,1676444031550 2023-02-15 06:53:53,721 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:53:53,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-15 06:53:53,729 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46669,1676444031550 2023-02-15 06:53:53,731 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.hbase-snapshot/.tmp 2023-02-15 06:53:53,744 INFO [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(952): ClusterId : e60c3a26-6484-40bb-86cd-5e8a1f308e92 2023-02-15 06:53:53,744 INFO [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(952): ClusterId : e60c3a26-6484-40bb-86cd-5e8a1f308e92 2023-02-15 06:53:53,744 INFO [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(952): ClusterId : e60c3a26-6484-40bb-86cd-5e8a1f308e92 2023-02-15 06:53:53,749 DEBUG [RS:0;jenkins-hbase4:35089] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-15 06:53:53,749 DEBUG [RS:1;jenkins-hbase4:40475] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-15 06:53:53,749 DEBUG [RS:2;jenkins-hbase4:41759] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-15 06:53:53,757 DEBUG [RS:0;jenkins-hbase4:35089] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-15 06:53:53,757 DEBUG [RS:2;jenkins-hbase4:41759] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-15 06:53:53,757 DEBUG [RS:1;jenkins-hbase4:40475] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-15 06:53:53,757 DEBUG [RS:2;jenkins-hbase4:41759] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-15 06:53:53,757 DEBUG [RS:0;jenkins-hbase4:35089] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-15 06:53:53,757 DEBUG [RS:1;jenkins-hbase4:40475] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-15 06:53:53,761 DEBUG [RS:1;jenkins-hbase4:40475] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-15 06:53:53,761 DEBUG [RS:2;jenkins-hbase4:41759] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-15 06:53:53,761 DEBUG [RS:0;jenkins-hbase4:35089] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-15 06:53:53,763 DEBUG [RS:0;jenkins-hbase4:35089] zookeeper.ReadOnlyZKClient(139): Connect 0x01e7a4d7 to 127.0.0.1:51507 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-15 06:53:53,763 DEBUG [RS:2;jenkins-hbase4:41759] zookeeper.ReadOnlyZKClient(139): Connect 0x392cd4d9 to 127.0.0.1:51507 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-15 06:53:53,763 DEBUG [RS:1;jenkins-hbase4:40475] zookeeper.ReadOnlyZKClient(139): Connect 0x5cd38ce0 to 127.0.0.1:51507 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-15 06:53:53,777 DEBUG [RS:0;jenkins-hbase4:35089] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@f205a9b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-15 06:53:53,778 DEBUG [RS:0;jenkins-hbase4:35089] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@b1fcfa7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-15 06:53:53,780 DEBUG [RS:1;jenkins-hbase4:40475] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5214790a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-15 06:53:53,780 DEBUG [RS:1;jenkins-hbase4:40475] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@562be790, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-15 06:53:53,782 DEBUG [RS:2;jenkins-hbase4:41759] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37eb2d8c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-15 06:53:53,782 DEBUG [RS:2;jenkins-hbase4:41759] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@33d4bfa9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-15 06:53:53,802 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:35089 2023-02-15 06:53:53,805 DEBUG [RS:1;jenkins-hbase4:40475] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:40475 2023-02-15 06:53:53,806 INFO [RS:0;jenkins-hbase4:35089] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-15 06:53:53,806 INFO [RS:1;jenkins-hbase4:40475] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-15 06:53:53,806 INFO [RS:0;jenkins-hbase4:35089] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-15 06:53:53,806 INFO [RS:1;jenkins-hbase4:40475] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-15 06:53:53,806 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1023): About to register with Master. 2023-02-15 06:53:53,806 DEBUG [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(1023): About to register with Master. 2023-02-15 06:53:53,807 DEBUG [RS:2;jenkins-hbase4:41759] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:41759 2023-02-15 06:53:53,807 INFO [RS:2;jenkins-hbase4:41759] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-15 06:53:53,807 INFO [RS:2;jenkins-hbase4:41759] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-15 06:53:53,808 DEBUG [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1023): About to register with Master. 2023-02-15 06:53:53,809 INFO [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46669,1676444031550 with isa=jenkins-hbase4.apache.org/172.31.14.131:35089, startcode=1676444032716 2023-02-15 06:53:53,809 INFO [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46669,1676444031550 with isa=jenkins-hbase4.apache.org/172.31.14.131:41759, startcode=1676444032818 2023-02-15 06:53:53,809 INFO [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46669,1676444031550 with isa=jenkins-hbase4.apache.org/172.31.14.131:40475, startcode=1676444032771 2023-02-15 06:53:53,829 DEBUG [RS:0;jenkins-hbase4:35089] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-15 06:53:53,829 DEBUG [RS:2;jenkins-hbase4:41759] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-15 06:53:53,829 DEBUG [RS:1;jenkins-hbase4:40475] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-15 06:53:53,876 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-15 06:53:53,883 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52159, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-15 06:53:53,883 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60679, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-15 06:53:53,883 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55275, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-15 06:53:53,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-15 06:53:53,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-15 06:53:53,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-15 06:53:53,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-15 06:53:53,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-15 06:53:53,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:53,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-15 06:53:53,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:53,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444063892 2023-02-15 06:53:53,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46669] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-15 06:53:53,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46669] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-15 06:53:53,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-15 06:53:53,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-15 06:53:53,900 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-15 06:53:53,900 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-15 06:53:53,906 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-15 06:53:53,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-15 06:53:53,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-15 06:53:53,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-15 06:53:53,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-15 06:53:53,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-15 06:53:53,918 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:53,924 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-15 06:53:53,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-15 06:53:53,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-15 06:53:53,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-15 06:53:53,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-15 06:53:53,928 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(2829): Master is not running yet 2023-02-15 06:53:53,928 DEBUG [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(2829): Master is not running yet 2023-02-15 06:53:53,929 WARN [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-15 06:53:53,928 DEBUG [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(2829): Master is not running yet 2023-02-15 06:53:53,929 WARN [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-15 06:53:53,929 WARN [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-15 06:53:53,929 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676444033929,5,FailOnTimeoutGroup] 2023-02-15 06:53:53,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676444033930,5,FailOnTimeoutGroup] 2023-02-15 06:53:53,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:53,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-15 06:53:53,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:53,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:53,951 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-15 06:53:53,952 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-15 06:53:53,952 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2 2023-02-15 06:53:53,973 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:53:53,975 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-15 06:53:53,977 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/info 2023-02-15 06:53:53,978 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-15 06:53:53,979 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:53,979 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-15 06:53:53,981 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/rep_barrier 2023-02-15 06:53:53,982 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-15 06:53:53,983 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:53,983 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-15 06:53:53,985 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/table 2023-02-15 06:53:53,985 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-15 06:53:53,986 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:53,987 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740 2023-02-15 06:53:53,988 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740 2023-02-15 06:53:53,991 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-15 06:53:53,993 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-15 06:53:53,996 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:53:53,996 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62072558, jitterRate=-0.07504680752754211}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-15 06:53:53,997 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: 2023-02-15 06:53:53,997 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-15 06:53:53,997 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-15 06:53:53,997 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-15 06:53:53,997 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-15 06:53:53,997 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-15 06:53:53,998 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-15 06:53:53,998 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: 2023-02-15 06:53:54,003 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-15 06:53:54,003 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-15 06:53:54,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-15 06:53:54,024 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-15 06:53:54,025 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-15 06:53:54,030 INFO [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46669,1676444031550 with isa=jenkins-hbase4.apache.org/172.31.14.131:35089, startcode=1676444032716 2023-02-15 06:53:54,030 INFO [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46669,1676444031550 with isa=jenkins-hbase4.apache.org/172.31.14.131:41759, startcode=1676444032818 2023-02-15 06:53:54,030 INFO [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46669,1676444031550 with isa=jenkins-hbase4.apache.org/172.31.14.131:40475, startcode=1676444032771 2023-02-15 06:53:54,035 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:54,035 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46669] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:54,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46669] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:54,042 DEBUG [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2 2023-02-15 06:53:54,042 DEBUG [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2 2023-02-15 06:53:54,042 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2 2023-02-15 06:53:54,042 DEBUG [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41175 2023-02-15 06:53:54,042 DEBUG [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41175 2023-02-15 06:53:54,042 DEBUG [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-15 06:53:54,042 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41175 2023-02-15 06:53:54,042 DEBUG [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-15 06:53:54,042 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-15 06:53:54,084 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-15 06:53:54,177 DEBUG [jenkins-hbase4:46669] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-15 06:53:54,181 DEBUG [jenkins-hbase4:46669] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:53:54,186 DEBUG [jenkins-hbase4:46669] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:53:54,187 DEBUG [jenkins-hbase4:46669] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:53:54,187 DEBUG [jenkins-hbase4:46669] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:53:54,187 DEBUG [jenkins-hbase4:46669] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:53:54,189 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41759,1676444032818, state=OPENING 2023-02-15 06:53:54,254 DEBUG [RS:2;jenkins-hbase4:41759] zookeeper.ZKUtil(162): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:54,254 WARN [RS:2;jenkins-hbase4:41759] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-15 06:53:54,254 INFO [RS:2;jenkins-hbase4:41759] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-15 06:53:54,254 DEBUG [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:54,254 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,35089,1676444032716] 2023-02-15 06:53:54,254 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41759,1676444032818] 2023-02-15 06:53:54,254 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40475,1676444032771] 2023-02-15 06:53:54,276 DEBUG [RS:1;jenkins-hbase4:40475] zookeeper.ZKUtil(162): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:54,277 DEBUG [RS:0;jenkins-hbase4:35089] zookeeper.ZKUtil(162): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:54,277 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-15 06:53:54,277 WARN [RS:0;jenkins-hbase4:35089] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-15 06:53:54,277 WARN [RS:1;jenkins-hbase4:40475] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-15 06:53:54,277 INFO [RS:0;jenkins-hbase4:35089] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-15 06:53:54,278 INFO [RS:1;jenkins-hbase4:40475] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-15 06:53:54,278 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:54,278 DEBUG [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:54,307 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:53:54,308 DEBUG [RS:2;jenkins-hbase4:41759] zookeeper.ZKUtil(162): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:54,308 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-15 06:53:54,308 DEBUG [RS:0;jenkins-hbase4:35089] zookeeper.ZKUtil(162): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:54,309 DEBUG [RS:2;jenkins-hbase4:41759] zookeeper.ZKUtil(162): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:54,309 DEBUG [RS:1;jenkins-hbase4:40475] zookeeper.ZKUtil(162): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:54,309 DEBUG [RS:0;jenkins-hbase4:35089] zookeeper.ZKUtil(162): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:54,309 DEBUG [RS:2;jenkins-hbase4:41759] zookeeper.ZKUtil(162): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:54,309 DEBUG [RS:1;jenkins-hbase4:40475] zookeeper.ZKUtil(162): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:54,310 DEBUG [RS:0;jenkins-hbase4:35089] zookeeper.ZKUtil(162): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:54,310 DEBUG [RS:1;jenkins-hbase4:40475] zookeeper.ZKUtil(162): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:54,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:53:54,320 DEBUG [RS:1;jenkins-hbase4:40475] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-15 06:53:54,320 DEBUG [RS:2;jenkins-hbase4:41759] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-15 06:53:54,320 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-15 06:53:54,330 INFO [RS:1;jenkins-hbase4:40475] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-15 06:53:54,330 INFO [RS:2;jenkins-hbase4:41759] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-15 06:53:54,330 INFO [RS:0;jenkins-hbase4:35089] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-15 06:53:54,349 INFO [RS:1;jenkins-hbase4:40475] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-15 06:53:54,349 INFO [RS:0;jenkins-hbase4:35089] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-15 06:53:54,349 INFO [RS:2;jenkins-hbase4:41759] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-15 06:53:54,353 INFO [RS:0;jenkins-hbase4:35089] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-15 06:53:54,353 INFO [RS:1;jenkins-hbase4:40475] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-15 06:53:54,353 INFO [RS:2;jenkins-hbase4:41759] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-15 06:53:54,354 INFO [RS:1;jenkins-hbase4:40475] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,354 INFO [RS:0;jenkins-hbase4:35089] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,354 INFO [RS:2;jenkins-hbase4:41759] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,355 INFO [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-15 06:53:54,355 INFO [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-15 06:53:54,355 INFO [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-15 06:53:54,363 INFO [RS:0;jenkins-hbase4:35089] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,363 INFO [RS:1;jenkins-hbase4:40475] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,363 INFO [RS:2;jenkins-hbase4:41759] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,363 DEBUG [RS:0;jenkins-hbase4:35089] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,363 DEBUG [RS:1;jenkins-hbase4:40475] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,363 DEBUG [RS:0;jenkins-hbase4:35089] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,363 DEBUG [RS:1;jenkins-hbase4:40475] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,363 DEBUG [RS:0;jenkins-hbase4:35089] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,363 DEBUG [RS:1;jenkins-hbase4:40475] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,363 DEBUG [RS:0;jenkins-hbase4:35089] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,363 DEBUG [RS:1;jenkins-hbase4:40475] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:0;jenkins-hbase4:35089] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:1;jenkins-hbase4:40475] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:0;jenkins-hbase4:35089] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-15 06:53:54,364 DEBUG [RS:1;jenkins-hbase4:40475] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-15 06:53:54,364 DEBUG [RS:0;jenkins-hbase4:35089] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:1;jenkins-hbase4:40475] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,363 DEBUG [RS:2;jenkins-hbase4:41759] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:1;jenkins-hbase4:40475] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:2;jenkins-hbase4:41759] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:1;jenkins-hbase4:40475] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:2;jenkins-hbase4:41759] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:1;jenkins-hbase4:40475] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:2;jenkins-hbase4:41759] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:0;jenkins-hbase4:35089] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,364 DEBUG [RS:2;jenkins-hbase4:41759] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,365 DEBUG [RS:0;jenkins-hbase4:35089] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,365 DEBUG [RS:2;jenkins-hbase4:41759] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-15 06:53:54,365 DEBUG [RS:0;jenkins-hbase4:35089] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,365 DEBUG [RS:2;jenkins-hbase4:41759] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,365 DEBUG [RS:2;jenkins-hbase4:41759] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,365 DEBUG [RS:2;jenkins-hbase4:41759] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,365 DEBUG [RS:2;jenkins-hbase4:41759] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-15 06:53:54,368 INFO [RS:1;jenkins-hbase4:40475] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,368 INFO [RS:1;jenkins-hbase4:40475] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,368 INFO [RS:1;jenkins-hbase4:40475] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,370 INFO [RS:0;jenkins-hbase4:35089] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,370 INFO [RS:0;jenkins-hbase4:35089] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,370 INFO [RS:0;jenkins-hbase4:35089] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,376 INFO [RS:2;jenkins-hbase4:41759] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,376 INFO [RS:2;jenkins-hbase4:41759] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,376 INFO [RS:2;jenkins-hbase4:41759] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,386 INFO [RS:0;jenkins-hbase4:35089] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-15 06:53:54,387 INFO [RS:1;jenkins-hbase4:40475] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-15 06:53:54,389 INFO [RS:2;jenkins-hbase4:41759] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-15 06:53:54,389 INFO [RS:2;jenkins-hbase4:41759] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41759,1676444032818-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,389 INFO [RS:0;jenkins-hbase4:35089] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35089,1676444032716-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,389 INFO [RS:1;jenkins-hbase4:40475] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40475,1676444032771-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,410 INFO [RS:1;jenkins-hbase4:40475] regionserver.Replication(203): jenkins-hbase4.apache.org,40475,1676444032771 started 2023-02-15 06:53:54,410 INFO [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,40475,1676444032771, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40475, sessionid=0x10056c1c0920002 2023-02-15 06:53:54,413 INFO [RS:2;jenkins-hbase4:41759] regionserver.Replication(203): jenkins-hbase4.apache.org,41759,1676444032818 started 2023-02-15 06:53:54,413 DEBUG [RS:1;jenkins-hbase4:40475] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-15 06:53:54,414 INFO [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,41759,1676444032818, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41759, sessionid=0x10056c1c0920003 2023-02-15 06:53:54,414 INFO [RS:0;jenkins-hbase4:35089] regionserver.Replication(203): jenkins-hbase4.apache.org,35089,1676444032716 started 2023-02-15 06:53:54,414 DEBUG [RS:2;jenkins-hbase4:41759] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-15 06:53:54,414 INFO [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,35089,1676444032716, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:35089, sessionid=0x10056c1c0920001 2023-02-15 06:53:54,414 DEBUG [RS:1;jenkins-hbase4:40475] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:54,415 DEBUG [RS:0;jenkins-hbase4:35089] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-15 06:53:54,414 DEBUG [RS:2;jenkins-hbase4:41759] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:54,415 DEBUG [RS:0;jenkins-hbase4:35089] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:54,416 DEBUG [RS:0;jenkins-hbase4:35089] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35089,1676444032716' 2023-02-15 06:53:54,415 DEBUG [RS:1;jenkins-hbase4:40475] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40475,1676444032771' 2023-02-15 06:53:54,416 DEBUG [RS:0;jenkins-hbase4:35089] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:54,415 DEBUG [RS:2;jenkins-hbase4:41759] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41759,1676444032818' 2023-02-15 06:53:54,416 DEBUG [RS:1;jenkins-hbase4:40475] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:54,416 DEBUG [RS:2;jenkins-hbase4:41759] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:54,417 DEBUG [RS:0;jenkins-hbase4:35089] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:54,417 DEBUG [RS:1;jenkins-hbase4:40475] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:54,417 DEBUG [RS:2;jenkins-hbase4:41759] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:54,417 DEBUG [RS:0;jenkins-hbase4:35089] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-15 06:53:54,417 DEBUG [RS:0;jenkins-hbase4:35089] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-15 06:53:54,417 DEBUG [RS:1;jenkins-hbase4:40475] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-15 06:53:54,417 DEBUG [RS:0;jenkins-hbase4:35089] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:54,418 DEBUG [RS:2;jenkins-hbase4:41759] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-15 06:53:54,418 DEBUG [RS:2;jenkins-hbase4:41759] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-15 06:53:54,418 DEBUG [RS:1;jenkins-hbase4:40475] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-15 06:53:54,418 DEBUG [RS:2;jenkins-hbase4:41759] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:54,418 DEBUG [RS:0;jenkins-hbase4:35089] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,35089,1676444032716' 2023-02-15 06:53:54,420 DEBUG [RS:2;jenkins-hbase4:41759] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41759,1676444032818' 2023-02-15 06:53:54,420 DEBUG [RS:1;jenkins-hbase4:40475] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:54,421 DEBUG [RS:1;jenkins-hbase4:40475] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40475,1676444032771' 2023-02-15 06:53:54,421 DEBUG [RS:1;jenkins-hbase4:40475] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-15 06:53:54,420 DEBUG [RS:2;jenkins-hbase4:41759] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-15 06:53:54,420 DEBUG [RS:0;jenkins-hbase4:35089] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-15 06:53:54,421 DEBUG [RS:1;jenkins-hbase4:40475] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-15 06:53:54,421 DEBUG [RS:2;jenkins-hbase4:41759] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-15 06:53:54,421 DEBUG [RS:0;jenkins-hbase4:35089] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-15 06:53:54,422 DEBUG [RS:1;jenkins-hbase4:40475] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-15 06:53:54,422 INFO [RS:1;jenkins-hbase4:40475] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-15 06:53:54,422 INFO [RS:1;jenkins-hbase4:40475] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-15 06:53:54,422 DEBUG [RS:2;jenkins-hbase4:41759] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-15 06:53:54,422 DEBUG [RS:0;jenkins-hbase4:35089] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-15 06:53:54,422 INFO [RS:2;jenkins-hbase4:41759] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-15 06:53:54,423 INFO [RS:0;jenkins-hbase4:35089] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-15 06:53:54,423 INFO [RS:0;jenkins-hbase4:35089] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-15 06:53:54,423 INFO [RS:2;jenkins-hbase4:41759] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-15 06:53:54,495 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:54,497 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-15 06:53:54,500 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42120, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-15 06:53:54,511 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-15 06:53:54,511 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-15 06:53:54,514 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41759%2C1676444032818.meta, suffix=.meta, logDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,41759,1676444032818, archiveDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/oldWALs, maxLogs=32 2023-02-15 06:53:54,533 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46725,DS-a8cfc2f8-a466-4675-bf87-0687f8b45116,DISK] 2023-02-15 06:53:54,533 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33311,DS-7f6910ca-5199-4aec-9cda-9050c12d81d5,DISK] 2023-02-15 06:53:54,533 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38215,DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36,DISK] 2023-02-15 06:53:54,536 INFO [RS:0;jenkins-hbase4:35089] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35089%2C1676444032716, suffix=, logDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,35089,1676444032716, archiveDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/oldWALs, maxLogs=32 2023-02-15 06:53:54,536 INFO [RS:2;jenkins-hbase4:41759] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41759%2C1676444032818, suffix=, logDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,41759,1676444032818, archiveDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/oldWALs, maxLogs=32 2023-02-15 06:53:54,537 INFO [RS:1;jenkins-hbase4:40475] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40475%2C1676444032771, suffix=, logDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,40475,1676444032771, archiveDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/oldWALs, maxLogs=32 2023-02-15 06:53:54,545 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,41759,1676444032818/jenkins-hbase4.apache.org%2C41759%2C1676444032818.meta.1676444034516.meta 2023-02-15 06:53:54,545 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38215,DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36,DISK], DatanodeInfoWithStorage[127.0.0.1:46725,DS-a8cfc2f8-a466-4675-bf87-0687f8b45116,DISK], DatanodeInfoWithStorage[127.0.0.1:33311,DS-7f6910ca-5199-4aec-9cda-9050c12d81d5,DISK]] 2023-02-15 06:53:54,546 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:53:54,554 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:53:54,555 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46725,DS-a8cfc2f8-a466-4675-bf87-0687f8b45116,DISK] 2023-02-15 06:53:54,555 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33311,DS-7f6910ca-5199-4aec-9cda-9050c12d81d5,DISK] 2023-02-15 06:53:54,556 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38215,DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36,DISK] 2023-02-15 06:53:54,556 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-15 06:53:54,566 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33311,DS-7f6910ca-5199-4aec-9cda-9050c12d81d5,DISK] 2023-02-15 06:53:54,573 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46725,DS-a8cfc2f8-a466-4675-bf87-0687f8b45116,DISK] 2023-02-15 06:53:54,573 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38215,DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36,DISK] 2023-02-15 06:53:54,576 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38215,DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36,DISK] 2023-02-15 06:53:54,576 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33311,DS-7f6910ca-5199-4aec-9cda-9050c12d81d5,DISK] 2023-02-15 06:53:54,576 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46725,DS-a8cfc2f8-a466-4675-bf87-0687f8b45116,DISK] 2023-02-15 06:53:54,577 INFO [RS:0;jenkins-hbase4:35089] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,35089,1676444032716/jenkins-hbase4.apache.org%2C35089%2C1676444032716.1676444034537 2023-02-15 06:53:54,580 DEBUG [RS:0;jenkins-hbase4:35089] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46725,DS-a8cfc2f8-a466-4675-bf87-0687f8b45116,DISK], DatanodeInfoWithStorage[127.0.0.1:33311,DS-7f6910ca-5199-4aec-9cda-9050c12d81d5,DISK], DatanodeInfoWithStorage[127.0.0.1:38215,DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36,DISK]] 2023-02-15 06:53:54,586 INFO [RS:2;jenkins-hbase4:41759] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,41759,1676444032818/jenkins-hbase4.apache.org%2C41759%2C1676444032818.1676444034538 2023-02-15 06:53:54,588 DEBUG [RS:2;jenkins-hbase4:41759] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33311,DS-7f6910ca-5199-4aec-9cda-9050c12d81d5,DISK], DatanodeInfoWithStorage[127.0.0.1:46725,DS-a8cfc2f8-a466-4675-bf87-0687f8b45116,DISK], DatanodeInfoWithStorage[127.0.0.1:38215,DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36,DISK]] 2023-02-15 06:53:54,589 INFO [RS:1;jenkins-hbase4:40475] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/WALs/jenkins-hbase4.apache.org,40475,1676444032771/jenkins-hbase4.apache.org%2C40475%2C1676444032771.1676444034539 2023-02-15 06:53:54,589 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-15 06:53:54,590 DEBUG [RS:1;jenkins-hbase4:40475] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38215,DS-1087c6ef-6c97-4c44-b142-54bf1c52cb36,DISK], DatanodeInfoWithStorage[127.0.0.1:46725,DS-a8cfc2f8-a466-4675-bf87-0687f8b45116,DISK], DatanodeInfoWithStorage[127.0.0.1:33311,DS-7f6910ca-5199-4aec-9cda-9050c12d81d5,DISK]] 2023-02-15 06:53:54,593 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-15 06:53:54,598 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-15 06:53:54,598 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:53:54,598 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-15 06:53:54,598 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-15 06:53:54,601 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-15 06:53:54,603 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/info 2023-02-15 06:53:54,604 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/info 2023-02-15 06:53:54,605 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-15 06:53:54,606 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:54,606 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-15 06:53:54,608 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/rep_barrier 2023-02-15 06:53:54,608 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/rep_barrier 2023-02-15 06:53:54,608 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-15 06:53:54,609 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:54,609 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-15 06:53:54,610 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/table 2023-02-15 06:53:54,610 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/table 2023-02-15 06:53:54,611 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-15 06:53:54,611 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:54,613 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740 2023-02-15 06:53:54,615 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740 2023-02-15 06:53:54,618 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-15 06:53:54,620 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-15 06:53:54,621 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65414019, jitterRate=-0.02525515854358673}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-15 06:53:54,622 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: 2023-02-15 06:53:54,630 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676444034487 2023-02-15 06:53:54,647 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-15 06:53:54,648 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-15 06:53:54,648 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,41759,1676444032818, state=OPEN 2023-02-15 06:53:54,651 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-15 06:53:54,651 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-15 06:53:54,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-15 06:53:54,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,41759,1676444032818 in 339 msec 2023-02-15 06:53:54,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-15 06:53:54,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 647 msec 2023-02-15 06:53:54,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 873 msec 2023-02-15 06:53:54,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1676444034666, completionTime=-1 2023-02-15 06:53:54,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-15 06:53:54,667 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:53:54,730 DEBUG [hconnection-0x4c388f12-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-15 06:53:54,733 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42134, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-15 06:53:54,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:53:54,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444094748 2023-02-15 06:53:54,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444154748 2023-02-15 06:53:54,748 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 81 msec 2023-02-15 06:53:54,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46669,1676444031550-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46669,1676444031550-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46669,1676444031550-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,772 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:46669, period=300000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,772 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-15 06:53:54,778 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-15 06:53:54,787 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-15 06:53:54,788 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-15 06:53:54,797 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-15 06:53:54,799 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:53:54,801 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:53:54,826 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42 2023-02-15 06:53:54,828 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42 empty. 2023-02-15 06:53:54,829 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42 2023-02-15 06:53:54,829 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-15 06:53:54,867 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-15 06:53:54,869 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 03b216dbc5303ada2e264ea03896de42, NAME => 'hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:53:54,888 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:53:54,888 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 03b216dbc5303ada2e264ea03896de42, disabling compactions & flushes 2023-02-15 06:53:54,888 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:53:54,888 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:53:54,888 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. after waiting 0 ms 2023-02-15 06:53:54,888 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:53:54,889 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:53:54,889 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 03b216dbc5303ada2e264ea03896de42: 2023-02-15 06:53:54,893 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:53:54,907 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676444034895"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444034895"}]},"ts":"1676444034895"} 2023-02-15 06:53:54,935 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:53:54,937 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:53:54,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444034937"}]},"ts":"1676444034937"} 2023-02-15 06:53:54,945 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-15 06:53:54,950 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:53:54,952 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:53:54,952 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:53:54,952 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:53:54,952 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:53:54,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=03b216dbc5303ada2e264ea03896de42, ASSIGN}] 2023-02-15 06:53:54,957 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=03b216dbc5303ada2e264ea03896de42, ASSIGN 2023-02-15 06:53:54,959 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=03b216dbc5303ada2e264ea03896de42, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35089,1676444032716; forceNewPlan=false, retain=false 2023-02-15 06:53:55,111 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:53:55,112 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=03b216dbc5303ada2e264ea03896de42, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:55,113 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676444035112"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444035112"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444035112"}]},"ts":"1676444035112"} 2023-02-15 06:53:55,119 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 03b216dbc5303ada2e264ea03896de42, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:53:55,273 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:55,273 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-15 06:53:55,276 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54396, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-15 06:53:55,281 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:53:55,282 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 03b216dbc5303ada2e264ea03896de42, NAME => 'hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:53:55,283 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:53:55,284 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 03b216dbc5303ada2e264ea03896de42 2023-02-15 06:53:55,284 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:53:55,284 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 03b216dbc5303ada2e264ea03896de42 2023-02-15 06:53:55,284 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 03b216dbc5303ada2e264ea03896de42 2023-02-15 06:53:55,286 INFO [StoreOpener-03b216dbc5303ada2e264ea03896de42-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 03b216dbc5303ada2e264ea03896de42 2023-02-15 06:53:55,288 DEBUG [StoreOpener-03b216dbc5303ada2e264ea03896de42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42/info 2023-02-15 06:53:55,288 DEBUG [StoreOpener-03b216dbc5303ada2e264ea03896de42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42/info 2023-02-15 06:53:55,288 INFO [StoreOpener-03b216dbc5303ada2e264ea03896de42-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 03b216dbc5303ada2e264ea03896de42 columnFamilyName info 2023-02-15 06:53:55,289 INFO [StoreOpener-03b216dbc5303ada2e264ea03896de42-1] regionserver.HStore(310): Store=03b216dbc5303ada2e264ea03896de42/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:55,291 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42 2023-02-15 06:53:55,291 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42 2023-02-15 06:53:55,295 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 03b216dbc5303ada2e264ea03896de42 2023-02-15 06:53:55,298 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:53:55,299 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 03b216dbc5303ada2e264ea03896de42; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62276561, jitterRate=-0.07200692594051361}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-15 06:53:55,299 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 03b216dbc5303ada2e264ea03896de42: 2023-02-15 06:53:55,301 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42., pid=6, masterSystemTime=1676444035273 2023-02-15 06:53:55,306 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:53:55,306 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:53:55,308 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:55,308 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676444035307"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444035307"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444035307"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444035307"}]},"ts":"1676444035307"} 2023-02-15 06:53:55,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-15 06:53:55,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 03b216dbc5303ada2e264ea03896de42, server=jenkins-hbase4.apache.org,35089,1676444032716 in 194 msec 2023-02-15 06:53:55,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-15 06:53:55,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=03b216dbc5303ada2e264ea03896de42, ASSIGN in 362 msec 2023-02-15 06:53:55,322 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:53:55,324 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444035324"}]},"ts":"1676444035324"} 2023-02-15 06:53:55,326 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-15 06:53:55,330 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:53:55,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 541 msec 2023-02-15 06:53:55,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-15 06:53:55,402 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-15 06:53:55,402 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:53:55,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-15 06:53:55,425 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54402, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-15 06:53:55,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-15 06:53:55,455 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-15 06:53:55,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 31 msec 2023-02-15 06:53:55,470 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:53:55,480 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-15 06:53:55,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 14 msec 2023-02-15 06:53:55,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-15 06:53:55,505 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-15 06:53:55,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.626sec 2023-02-15 06:53:55,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-15 06:53:55,511 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-15 06:53:55,511 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-15 06:53:55,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46669,1676444031550-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-15 06:53:55,514 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46669,1676444031550-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-15 06:53:55,522 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-02-15 06:53:55,550 DEBUG [Listener at localhost/37669] zookeeper.ReadOnlyZKClient(139): Connect 0x37e887de to 127.0.0.1:51507 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-15 06:53:55,556 DEBUG [Listener at localhost/37669] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c8d5aec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-15 06:53:55,574 DEBUG [hconnection-0x7e2f5f25-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-15 06:53:55,587 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42146, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-15 06:53:55,598 INFO [Listener at localhost/37669] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,46669,1676444031550 2023-02-15 06:53:55,633 INFO [Listener at localhost/37669] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=358, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=245, ProcessCount=168, AvailableMemoryMB=3360 2023-02-15 06:53:55,638 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:53:55,648 DEBUG [Listener at localhost/37669] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-15 06:53:55,651 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58654, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-15 06:53:55,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-15 06:53:55,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:53:55,668 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-15 06:53:55,675 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:53:55,678 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:53:55,681 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:53:55,684 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776 2023-02-15 06:53:55,685 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776 empty. 2023-02-15 06:53:55,688 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776 2023-02-15 06:53:55,688 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-15 06:53:55,711 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-15 06:53:55,713 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => e93940bab783f139204747313e792776, NAME => 'testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:53:55,732 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:53:55,732 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing e93940bab783f139204747313e792776, disabling compactions & flushes 2023-02-15 06:53:55,732 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:55,732 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:55,732 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. after waiting 0 ms 2023-02-15 06:53:55,732 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:55,732 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:55,732 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for e93940bab783f139204747313e792776: 2023-02-15 06:53:55,738 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:53:55,740 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676444035739"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444035739"}]},"ts":"1676444035739"} 2023-02-15 06:53:55,743 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:53:55,749 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:53:55,749 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444035749"}]},"ts":"1676444035749"} 2023-02-15 06:53:55,751 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-15 06:53:55,757 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:53:55,759 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:53:55,759 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:53:55,759 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:53:55,759 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:53:55,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e93940bab783f139204747313e792776, ASSIGN}] 2023-02-15 06:53:55,762 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e93940bab783f139204747313e792776, ASSIGN 2023-02-15 06:53:55,764 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e93940bab783f139204747313e792776, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:53:55,914 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:53:55,915 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=e93940bab783f139204747313e792776, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:55,916 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676444035915"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444035915"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444035915"}]},"ts":"1676444035915"} 2023-02-15 06:53:55,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure e93940bab783f139204747313e792776, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:53:56,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:56,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e93940bab783f139204747313e792776, NAME => 'testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:53:56,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:53:56,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter e93940bab783f139204747313e792776 2023-02-15 06:53:56,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:53:56,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e93940bab783f139204747313e792776 2023-02-15 06:53:56,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e93940bab783f139204747313e792776 2023-02-15 06:53:56,083 INFO [StoreOpener-e93940bab783f139204747313e792776-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e93940bab783f139204747313e792776 2023-02-15 06:53:56,086 DEBUG [StoreOpener-e93940bab783f139204747313e792776-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1 2023-02-15 06:53:56,086 DEBUG [StoreOpener-e93940bab783f139204747313e792776-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1 2023-02-15 06:53:56,087 INFO [StoreOpener-e93940bab783f139204747313e792776-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e93940bab783f139204747313e792776 columnFamilyName cf1 2023-02-15 06:53:56,088 INFO [StoreOpener-e93940bab783f139204747313e792776-1] regionserver.HStore(310): Store=e93940bab783f139204747313e792776/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:56,088 INFO [StoreOpener-e93940bab783f139204747313e792776-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e93940bab783f139204747313e792776 2023-02-15 06:53:56,090 DEBUG [StoreOpener-e93940bab783f139204747313e792776-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2 2023-02-15 06:53:56,090 DEBUG [StoreOpener-e93940bab783f139204747313e792776-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2 2023-02-15 06:53:56,091 INFO [StoreOpener-e93940bab783f139204747313e792776-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e93940bab783f139204747313e792776 columnFamilyName cf2 2023-02-15 06:53:56,091 INFO [StoreOpener-e93940bab783f139204747313e792776-1] regionserver.HStore(310): Store=e93940bab783f139204747313e792776/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:56,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776 2023-02-15 06:53:56,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776 2023-02-15 06:53:56,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:53:56,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e93940bab783f139204747313e792776 2023-02-15 06:53:56,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:53:56,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e93940bab783f139204747313e792776; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63425914, jitterRate=-0.054880231618881226}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:53:56,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e93940bab783f139204747313e792776: 2023-02-15 06:53:56,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776., pid=11, masterSystemTime=1676444036073 2023-02-15 06:53:56,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:56,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:56,108 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=e93940bab783f139204747313e792776, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,108 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676444036108"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444036108"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444036108"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444036108"}]},"ts":"1676444036108"} 2023-02-15 06:53:56,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-15 06:53:56,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure e93940bab783f139204747313e792776, server=jenkins-hbase4.apache.org,41759,1676444032818 in 192 msec 2023-02-15 06:53:56,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-15 06:53:56,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e93940bab783f139204747313e792776, ASSIGN in 355 msec 2023-02-15 06:53:56,119 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:53:56,120 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444036119"}]},"ts":"1676444036119"} 2023-02-15 06:53:56,122 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-15 06:53:56,127 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:53:56,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 454 msec 2023-02-15 06:53:56,204 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:53:56,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,216 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:56,216 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,217 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-15 06:53:56,217 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-15 06:53:56,218 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,218 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,220 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,220 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,220 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,221 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,222 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,222 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,222 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:56,222 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,222 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,222 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,225 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:56,226 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,228 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:56,228 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:56,228 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:56,228 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,228 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:56,229 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:56,228 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:56,229 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,230 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:56,230 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:56,230 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:56,232 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:53:56,232 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:56,232 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:53:56,233 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. started... 2023-02-15 06:53:56,234 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e93940bab783f139204747313e792776 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-15 06:53:56,234 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,239 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,239 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,239 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:56,240 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,240 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:56,312 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58 2023-02-15 06:53:56,422 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/48ac10727b6144f0bf91e9c72cba9ab5 2023-02-15 06:53:56,434 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58 2023-02-15 06:53:56,446 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:53:56,449 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/48ac10727b6144f0bf91e9c72cba9ab5 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5 2023-02-15 06:53:56,461 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:53:56,464 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e93940bab783f139204747313e792776 in 230ms, sequenceid=6, compaction requested=false 2023-02-15 06:53:56,466 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:56,469 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e93940bab783f139204747313e792776: 2023-02-15 06:53:56,469 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:56,469 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:53:56,470 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:53:56,470 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,470 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:56,470 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:56,472 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,472 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:56,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:56,473 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:56,473 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:56,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:56,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:56,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-15 06:53:56,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1248da7d[Count = 0] remaining members to acquire global barrier 2023-02-15 06:53:56,477 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-15 06:53:56,477 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,484 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,484 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,484 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,485 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:56,485 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,485 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:53:56,485 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:56,485 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:56,485 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,485 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:53:56,485 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:53:56,485 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:56,485 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:56,485 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:53:56,490 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:56,490 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,490 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:56,491 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:56,491 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,491 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:56,492 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,491 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:56,492 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,492 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:56,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:56,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:56,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:53:56,498 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-15 06:53:56,498 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:53:56,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:53:56,498 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:53:56,498 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,498 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:53:56,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:56,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:56,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,503 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:56,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,503 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,504 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:56,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,507 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:56,508 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,508 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,520 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,520 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,520 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,520 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,520 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,520 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,520 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,521 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,521 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:56,522 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:53:56,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,521 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,524 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,524 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,524 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,524 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,524 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,524 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,524 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,524 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,521 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,526 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:53:56,526 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:53:56,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,627 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:53:56,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:53:56,656 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:53:56,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:53:56,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,659 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:56,659 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,660 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-15 06:53:56,660 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-15 06:53:56,661 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,661 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,662 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,662 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,662 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,662 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,663 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,663 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:53:56,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,664 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:56,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,664 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:56,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:56,664 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:56,664 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,665 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,665 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:56,665 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,665 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,665 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:56,665 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,665 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:56,665 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,666 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:56,666 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:56,666 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:56,666 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:56,666 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:56,666 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:53:56,666 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:53:56,666 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:56,666 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,666 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. started... 2023-02-15 06:53:56,666 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:56,667 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,667 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:56,667 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:56,667 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e93940bab783f139204747313e792776 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:53:56,667 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:56,669 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,669 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,669 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:56,671 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,671 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,672 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:56,686 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37 2023-02-15 06:53:56,712 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/cea384b791b14b5baadc4badcb91a84c 2023-02-15 06:53:56,720 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37 2023-02-15 06:53:56,728 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:53:56,729 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/cea384b791b14b5baadc4badcb91a84c as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c 2023-02-15 06:53:56,737 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:53:56,738 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e93940bab783f139204747313e792776 in 71ms, sequenceid=19, compaction requested=false 2023-02-15 06:53:56,739 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e93940bab783f139204747313e792776: 2023-02-15 06:53:56,739 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:56,739 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:53:56,739 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:53:56,740 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,740 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:56,740 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:56,741 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,741 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:56,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:56,742 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,742 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:56,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:56,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:56,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:56,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-15 06:53:56,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2445d4a1[Count = 0] remaining members to acquire global barrier 2023-02-15 06:53:56,744 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-15 06:53:56,745 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,746 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,746 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,746 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,746 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:56,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,746 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:56,746 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,746 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:53:56,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,746 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:53:56,746 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:56,746 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:56,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,746 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:53:56,747 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:56,747 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:56,747 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:53:56,748 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:56,748 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,749 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:56,750 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:56,750 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:56,750 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,750 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,750 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,750 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:56,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:56,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:56,751 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:56,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:56,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:56,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:56,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:53:56,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:53:56,756 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-15 06:53:56,756 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:53:56,756 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:53:56,756 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,756 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:53:56,758 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,758 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:56,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:56,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,758 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,758 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:56,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,760 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,761 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:56,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,763 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:56,763 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,764 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,773 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,773 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,773 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,773 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,774 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,773 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,774 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,774 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:56,774 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:53:56,774 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:56,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,774 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,775 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,775 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:53:56,775 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,775 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:53:56,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,775 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:56,775 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:56,775 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,875 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:53:56,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:53:56,899 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:53:56,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:53:56,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,902 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:56,902 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,903 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-15 06:53:56,903 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-15 06:53:56,903 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,903 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,905 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,905 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,905 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,905 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:56,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:56,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:56,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,905 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,906 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:56,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,906 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:56,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:56,907 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:56,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:56,907 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:56,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:56,909 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:56,908 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:56,909 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:56,909 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:56,909 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:56,909 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:53:56,909 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:53:56,909 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:56,909 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. started... 2023-02-15 06:53:56,910 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e93940bab783f139204747313e792776 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:53:56,911 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,912 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,912 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,912 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:56,912 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,912 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:56,929 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/833ac1bdcea04bce9637c68a0d427b49 2023-02-15 06:53:56,953 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/bb691aacb0a94e62be83e5cd26097161 2023-02-15 06:53:56,962 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/833ac1bdcea04bce9637c68a0d427b49 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49 2023-02-15 06:53:56,970 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:53:56,971 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/bb691aacb0a94e62be83e5cd26097161 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161 2023-02-15 06:53:56,979 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:53:56,980 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e93940bab783f139204747313e792776 in 71ms, sequenceid=32, compaction requested=true 2023-02-15 06:53:56,981 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e93940bab783f139204747313e792776: 2023-02-15 06:53:56,981 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:56,981 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:53:56,982 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:53:56,982 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,982 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:56,982 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:56,984 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,984 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:56,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:56,985 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:56,985 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:56,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:56,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:56,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-15 06:53:56,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d2b0678[Count = 0] remaining members to acquire global barrier 2023-02-15 06:53:56,987 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-15 06:53:56,987 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,989 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,989 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,989 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:53:56,989 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,989 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:53:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:53:56,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:53:56,992 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:56,992 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,993 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:56,993 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:56,993 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,993 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:56,995 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:56,995 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,995 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:56,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,995 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:56,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:56,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:56,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:56,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:56,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:56,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:56,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:56,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:56,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:53:56,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:53:56,999 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-15 06:53:56,999 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:53:56,999 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:53:56,999 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:56,999 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:53:57,001 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,001 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:57,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:57,001 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,001 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,001 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,001 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,002 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,002 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,003 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:57,003 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:57,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,006 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,006 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,007 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,010 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,010 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:57,011 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:53:57,010 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,011 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,012 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:53:57,012 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,012 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:53:57,012 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,012 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,012 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,012 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,112 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:53:57,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:53:57,135 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:53:57,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:53:57,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,137 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:57,137 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,138 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-15 06:53:57,138 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-15 06:53:57,139 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,139 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,140 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,141 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,141 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:53:57,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,142 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,142 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,142 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:57,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:57,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:57,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,143 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:57,143 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,143 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,143 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,143 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:57,143 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:57,143 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,143 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,143 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:57,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:57,144 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:57,144 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:57,144 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:57,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,144 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,144 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:57,145 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,145 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:57,146 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:57,146 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:57,146 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:57,146 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:53:57,146 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:53:57,146 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:57,146 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. started... 2023-02-15 06:53:57,146 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e93940bab783f139204747313e792776 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:53:57,148 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,149 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,149 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,150 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:57,150 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,150 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:57,173 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/70482e4c98e14aaf9ec682064c496924 2023-02-15 06:53:57,214 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/a402ec297b3b4a808833bae0c78df052 2023-02-15 06:53:57,225 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/70482e4c98e14aaf9ec682064c496924 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924 2023-02-15 06:53:57,233 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:53:57,234 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/a402ec297b3b4a808833bae0c78df052 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052 2023-02-15 06:53:57,243 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:53:57,244 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e93940bab783f139204747313e792776 in 98ms, sequenceid=45, compaction requested=true 2023-02-15 06:53:57,244 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e93940bab783f139204747313e792776: 2023-02-15 06:53:57,244 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:57,244 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:53:57,245 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:53:57,245 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,245 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:57,245 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:57,250 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,250 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:57,251 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,251 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:57,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:57,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-15 06:53:57,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37139d83[Count = 0] remaining members to acquire global barrier 2023-02-15 06:53:57,253 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-15 06:53:57,253 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,254 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,254 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,254 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,255 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,255 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:53:57,255 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:57,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,255 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:57,255 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:57,255 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:57,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,255 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:53:57,255 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:57,255 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:57,255 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:53:57,255 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:53:57,257 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:57,257 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,257 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:57,258 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,258 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,258 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:57,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,260 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:57,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:57,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:57,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:57,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:53:57,265 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:53:57,265 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-15 06:53:57,265 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:53:57,265 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:53:57,265 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,265 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:53:57,269 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,269 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:57,270 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,270 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,269 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,270 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,270 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,270 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,271 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:57,272 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:57,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,273 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,274 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,274 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,278 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,278 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,278 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,278 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,278 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,278 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,278 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,278 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,278 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:57,279 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:53:57,278 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,279 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,279 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,279 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,279 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:53:57,280 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,280 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,280 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:53:57,280 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,280 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,280 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,280 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,380 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:53:57,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:53:57,411 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:53:57,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:53:57,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,414 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:57,414 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,414 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-15 06:53:57,415 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-15 06:53:57,415 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,415 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,416 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,416 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,417 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:57,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,418 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,419 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,420 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:57,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,421 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,422 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:57,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,422 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:57,422 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,422 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:53:57,422 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,422 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,423 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:57,425 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:57,425 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,425 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,426 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:57,426 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:57,426 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:57,426 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:57,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:57,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,427 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,427 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:53:57,427 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:53:57,427 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:57,427 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:57,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,427 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. started... 2023-02-15 06:53:57,427 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:57,427 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,429 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:57,429 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:57,429 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:57,429 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e93940bab783f139204747313e792776 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:53:57,431 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,438 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,438 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,438 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:57,438 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,439 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:57,452 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/68ae222995854654a4551bf9752f2cf4 2023-02-15 06:53:57,478 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/6c2de12cfabf497c95cabd50b8b8a777 2023-02-15 06:53:57,486 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/68ae222995854654a4551bf9752f2cf4 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/68ae222995854654a4551bf9752f2cf4 2023-02-15 06:53:57,493 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/68ae222995854654a4551bf9752f2cf4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:53:57,494 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/6c2de12cfabf497c95cabd50b8b8a777 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/6c2de12cfabf497c95cabd50b8b8a777 2023-02-15 06:53:57,501 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/6c2de12cfabf497c95cabd50b8b8a777, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:53:57,502 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e93940bab783f139204747313e792776 in 74ms, sequenceid=58, compaction requested=true 2023-02-15 06:53:57,502 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e93940bab783f139204747313e792776: 2023-02-15 06:53:57,502 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:57,503 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:53:57,503 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:53:57,503 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,503 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:57,503 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:57,505 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,506 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:57,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:57,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:57,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-15 06:53:57,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@635c222e[Count = 0] remaining members to acquire global barrier 2023-02-15 06:53:57,508 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-15 06:53:57,508 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,510 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,510 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,510 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,510 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,510 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:57,510 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:53:57,510 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:57,510 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:57,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,510 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,510 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:53:57,511 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:57,511 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:57,511 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:53:57,510 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:53:57,512 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,512 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:57,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:57,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:57,512 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,513 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:57,514 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:57,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:57,514 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,514 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:57,514 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:57,514 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,515 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:53:57,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:53:57,520 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-15 06:53:57,520 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:53:57,520 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:53:57,520 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,520 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:53:57,521 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,521 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:57,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,522 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,521 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,523 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,523 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,525 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:57,526 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,526 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,533 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,533 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,533 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,533 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,533 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,533 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,533 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,533 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,533 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,534 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,534 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,534 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,533 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:57,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,533 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,534 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:53:57,534 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,535 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,534 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,535 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,535 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:53:57,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,535 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:53:57,535 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,635 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:53:57,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:53:57,659 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:53:57,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:53:57,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,662 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:57,662 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,663 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-15 06:53:57,663 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-15 06:53:57,663 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,663 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,666 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,666 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,666 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,666 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,667 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,667 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:53:57,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:57,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:57,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:53:57,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,668 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:57,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,668 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,668 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:57,668 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:57,668 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,668 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-15 06:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:57,669 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,668 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-15 06:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:57,669 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:53:57,670 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:57,670 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:57,670 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-15 06:53:57,670 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:57,670 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:57,671 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:53:57,671 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:53:57,671 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:57,671 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. started... 2023-02-15 06:53:57,671 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e93940bab783f139204747313e792776 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:57,688 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/21c5747a134142ae90b63eb374381c53 2023-02-15 06:53:57,714 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/7ea249fb5ee4470bb66f9d13f8bb9022 2023-02-15 06:53:57,722 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/21c5747a134142ae90b63eb374381c53 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/21c5747a134142ae90b63eb374381c53 2023-02-15 06:53:57,730 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/21c5747a134142ae90b63eb374381c53, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:53:57,731 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/7ea249fb5ee4470bb66f9d13f8bb9022 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/7ea249fb5ee4470bb66f9d13f8bb9022 2023-02-15 06:53:57,739 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/7ea249fb5ee4470bb66f9d13f8bb9022, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:53:57,740 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e93940bab783f139204747313e792776 in 69ms, sequenceid=71, compaction requested=true 2023-02-15 06:53:57,740 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e93940bab783f139204747313e792776: 2023-02-15 06:53:57,740 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:57,740 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:53:57,742 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:53:57,742 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,742 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-15 06:53:57,742 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-15 06:53:57,744 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,744 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:57,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:57,745 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,745 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:53:57,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:57,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:57,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:57,747 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-15 06:53:57,747 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69361238[Count = 0] remaining members to acquire global barrier 2023-02-15 06:53:57,747 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-15 06:53:57,747 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,748 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,748 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,748 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,748 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:57,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,749 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:57,749 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-15 06:53:57,749 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:57,749 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:57,749 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-15 06:53:57,748 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,749 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:53:57,749 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:53:57,749 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:53:57,749 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:53:57,752 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,752 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,752 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:57,752 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:57,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:57,752 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:57,753 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,752 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-15 06:53:57,753 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:57,753 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,753 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-15 06:53:57,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:57,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:57,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:57,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:53:57,757 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-15 06:53:57,757 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:53:57,758 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:53:57,758 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,758 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:53:57,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:53:57,759 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,759 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:53:57,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:53:57,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,759 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,759 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,759 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,760 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:53:57,760 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,760 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,761 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:53:57,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,761 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:53:57,763 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,764 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,764 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,767 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,767 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,767 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,767 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,767 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,767 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:53:57,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,768 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-15 06:53:57,768 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:53:57,767 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,767 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,767 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,768 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:53:57,768 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:53:57,768 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:53:57,768 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:53:57,767 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:53:57,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:53:57,768 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:53:57,769 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,769 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:53:57,769 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:53:57,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:53:57,769 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,770 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-15 06:53:57,868 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:53:57,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:53:57,890 DEBUG [Listener at localhost/37669] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,897 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:57,898 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:57,898 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:57,899 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:57,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e93940bab783f139204747313e792776, UNASSIGN}] 2023-02-15 06:53:57,907 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e93940bab783f139204747313e792776, UNASSIGN 2023-02-15 06:53:57,908 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=e93940bab783f139204747313e792776, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:57,908 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676444037908"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444037908"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444037908"}]},"ts":"1676444037908"} 2023-02-15 06:53:57,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure e93940bab783f139204747313e792776, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:53:58,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e93940bab783f139204747313e792776 2023-02-15 06:53:58,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e93940bab783f139204747313e792776, disabling compactions & flushes 2023-02-15 06:53:58,071 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:58,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:58,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. after waiting 0 ms 2023-02-15 06:53:58,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:58,071 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing e93940bab783f139204747313e792776 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-15 06:53:58,089 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/eae719281a7b435cb97500bfdc1e9fb7 2023-02-15 06:53:58,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/83d9ab44f2fa429fa4da6eb1e9952c5a 2023-02-15 06:53:58,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf1/eae719281a7b435cb97500bfdc1e9fb7 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/eae719281a7b435cb97500bfdc1e9fb7 2023-02-15 06:53:58,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/eae719281a7b435cb97500bfdc1e9fb7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:53:58,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.tmp/cf2/83d9ab44f2fa429fa4da6eb1e9952c5a as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/83d9ab44f2fa429fa4da6eb1e9952c5a 2023-02-15 06:53:58,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/83d9ab44f2fa429fa4da6eb1e9952c5a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:53:58,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e93940bab783f139204747313e792776 in 67ms, sequenceid=82, compaction requested=true 2023-02-15 06:53:58,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-15 06:53:58,159 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:53:58,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. 2023-02-15 06:53:58,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e93940bab783f139204747313e792776: 2023-02-15 06:53:58,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e93940bab783f139204747313e792776 2023-02-15 06:53:58,163 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=e93940bab783f139204747313e792776, regionState=CLOSED 2023-02-15 06:53:58,164 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676444038163"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444038163"}]},"ts":"1676444038163"} 2023-02-15 06:53:58,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-15 06:53:58,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure e93940bab783f139204747313e792776, server=jenkins-hbase4.apache.org,41759,1676444032818 in 256 msec 2023-02-15 06:53:58,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-15 06:53:58,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e93940bab783f139204747313e792776, UNASSIGN in 263 msec 2023-02-15 06:53:58,190 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=e93940bab783f139204747313e792776, threads=14 2023-02-15 06:53:58,192 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,192 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/6c2de12cfabf497c95cabd50b8b8a777 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,192 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/7ea249fb5ee4470bb66f9d13f8bb9022 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,192 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/83d9ab44f2fa429fa4da6eb1e9952c5a for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,193 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,193 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,193 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,193 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/21c5747a134142ae90b63eb374381c53 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,194 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/68ae222995854654a4551bf9752f2cf4 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,194 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,194 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,194 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,195 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,200 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/eae719281a7b435cb97500bfdc1e9fb7 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,238 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/7ea249fb5ee4470bb66f9d13f8bb9022, top=true 2023-02-15 06:53:58,240 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/21c5747a134142ae90b63eb374381c53, top=true 2023-02-15 06:53:58,244 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/eae719281a7b435cb97500bfdc1e9fb7, top=true 2023-02-15 06:53:58,247 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/83d9ab44f2fa429fa4da6eb1e9952c5a, top=true 2023-02-15 06:53:58,248 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/6c2de12cfabf497c95cabd50b8b8a777, top=true 2023-02-15 06:53:58,252 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/68ae222995854654a4551bf9752f2cf4, top=true 2023-02-15 06:53:58,284 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.splits/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-68ae222995854654a4551bf9752f2cf4 for child: a670097e89a3ff97d31f6ef5bcbe626a, parent: e93940bab783f139204747313e792776 2023-02-15 06:53:58,284 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/68ae222995854654a4551bf9752f2cf4 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,296 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.splits/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-6c2de12cfabf497c95cabd50b8b8a777 for child: a670097e89a3ff97d31f6ef5bcbe626a, parent: e93940bab783f139204747313e792776 2023-02-15 06:53:58,296 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.splits/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-7ea249fb5ee4470bb66f9d13f8bb9022 for child: a670097e89a3ff97d31f6ef5bcbe626a, parent: e93940bab783f139204747313e792776 2023-02-15 06:53:58,296 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/6c2de12cfabf497c95cabd50b8b8a777 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,296 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.splits/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-83d9ab44f2fa429fa4da6eb1e9952c5a for child: a670097e89a3ff97d31f6ef5bcbe626a, parent: e93940bab783f139204747313e792776 2023-02-15 06:53:58,296 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/83d9ab44f2fa429fa4da6eb1e9952c5a for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,296 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.splits/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-eae719281a7b435cb97500bfdc1e9fb7 for child: a670097e89a3ff97d31f6ef5bcbe626a, parent: e93940bab783f139204747313e792776 2023-02-15 06:53:58,296 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/7ea249fb5ee4470bb66f9d13f8bb9022 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,296 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/eae719281a7b435cb97500bfdc1e9fb7 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,299 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/.splits/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-21c5747a134142ae90b63eb374381c53 for child: a670097e89a3ff97d31f6ef5bcbe626a, parent: e93940bab783f139204747313e792776 2023-02-15 06:53:58,299 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/21c5747a134142ae90b63eb374381c53 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,420 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,429 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,430 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,433 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,433 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,440 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,440 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,784 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49 for region: e93940bab783f139204747313e792776 2023-02-15 06:53:58,784 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region e93940bab783f139204747313e792776 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-15 06:53:58,839 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:53:58,841 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:53:58,844 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676444038844"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676444038844"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676444038844"}]},"ts":"1676444038844"} 2023-02-15 06:53:58,844 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676444038844"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444038844"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444038844"}]},"ts":"1676444038844"} 2023-02-15 06:53:58,845 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676444038844"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444038844"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444038844"}]},"ts":"1676444038844"} 2023-02-15 06:53:58,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c2a14c04f55c99754ea9727c4d88d1d8, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a670097e89a3ff97d31f6ef5bcbe626a, ASSIGN}] 2023-02-15 06:53:58,896 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a670097e89a3ff97d31f6ef5bcbe626a, ASSIGN 2023-02-15 06:53:58,897 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a670097e89a3ff97d31f6ef5bcbe626a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:53:58,898 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c2a14c04f55c99754ea9727c4d88d1d8, ASSIGN 2023-02-15 06:53:58,899 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c2a14c04f55c99754ea9727c4d88d1d8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:53:59,048 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-15 06:53:59,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=c2a14c04f55c99754ea9727c4d88d1d8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:59,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676444039050"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444039050"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444039050"}]},"ts":"1676444039050"} 2023-02-15 06:53:59,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure c2a14c04f55c99754ea9727c4d88d1d8, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:53:59,058 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=a670097e89a3ff97d31f6ef5bcbe626a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:59,059 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676444039058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444039058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444039058"}]},"ts":"1676444039058"} 2023-02-15 06:53:59,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure a670097e89a3ff97d31f6ef5bcbe626a, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:53:59,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:53:59,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c2a14c04f55c99754ea9727c4d88d1d8, NAME => 'testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.', STARTKEY => '', ENDKEY => '26'} 2023-02-15 06:53:59,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:53:59,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:53:59,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:53:59,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:53:59,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:53:59,213 INFO [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:53:59,214 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1 2023-02-15 06:53:59,214 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1 2023-02-15 06:53:59,215 INFO [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c2a14c04f55c99754ea9727c4d88d1d8 columnFamilyName cf1 2023-02-15 06:53:59,246 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924-bottom 2023-02-15 06:53:59,253 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49-bottom 2023-02-15 06:53:59,260 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37-bottom 2023-02-15 06:53:59,266 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58-bottom 2023-02-15 06:53:59,267 INFO [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(310): Store=c2a14c04f55c99754ea9727c4d88d1d8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:59,267 INFO [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:53:59,268 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2 2023-02-15 06:53:59,268 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2 2023-02-15 06:53:59,269 INFO [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c2a14c04f55c99754ea9727c4d88d1d8 columnFamilyName cf2 2023-02-15 06:53:59,283 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5-bottom 2023-02-15 06:53:59,289 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052-bottom 2023-02-15 06:53:59,295 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161-bottom 2023-02-15 06:53:59,302 DEBUG [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c-bottom 2023-02-15 06:53:59,302 INFO [StoreOpener-c2a14c04f55c99754ea9727c4d88d1d8-1] regionserver.HStore(310): Store=c2a14c04f55c99754ea9727c4d88d1d8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:59,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:53:59,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:53:59,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:53:59,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:53:59,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c2a14c04f55c99754ea9727c4d88d1d8; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68745384, jitterRate=0.024386048316955566}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:53:59,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c2a14c04f55c99754ea9727c4d88d1d8: 2023-02-15 06:53:59,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8., pid=17, masterSystemTime=1676444039206 2023-02-15 06:53:59,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:53:59,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:53:59,316 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-15 06:53:59,316 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-15 06:53:59,318 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:53:59,318 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:53:59,318 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1912): c2a14c04f55c99754ea9727c4d88d1d8/cf2 is initiating minor compaction (all files) 2023-02-15 06:53:59,318 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1912): c2a14c04f55c99754ea9727c4d88d1d8/cf1 is initiating minor compaction (all files) 2023-02-15 06:53:59,318 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c2a14c04f55c99754ea9727c4d88d1d8/cf1 in testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:53:59,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:53:59,318 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c2a14c04f55c99754ea9727c4d88d1d8/cf2 in testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:53:59,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:53:59,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:53:59,319 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=c2a14c04f55c99754ea9727c4d88d1d8, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:59,319 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924-bottom] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/.tmp, totalSize=19.9 K 2023-02-15 06:53:59,319 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052-bottom] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/.tmp, totalSize=19.9 K 2023-02-15 06:53:59,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a670097e89a3ff97d31f6ef5bcbe626a, NAME => 'testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.', STARTKEY => '26', ENDKEY => ''} 2023-02-15 06:53:59,320 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676444039319"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444039319"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444039319"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444039319"}]},"ts":"1676444039319"} 2023-02-15 06:53:59,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:53:59,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:59,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:53:59,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:59,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:59,322 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444036181 2023-02-15 06:53:59,323 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444036181 2023-02-15 06:53:59,323 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444036634 2023-02-15 06:53:59,324 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444036634 2023-02-15 06:53:59,325 INFO [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:59,324 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444036877 2023-02-15 06:53:59,326 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444036877 2023-02-15 06:53:59,326 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444037115 2023-02-15 06:53:59,326 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444037115 2023-02-15 06:53:59,326 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1 2023-02-15 06:53:59,327 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1 2023-02-15 06:53:59,328 INFO [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a670097e89a3ff97d31f6ef5bcbe626a columnFamilyName cf1 2023-02-15 06:53:59,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-15 06:53:59,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure c2a14c04f55c99754ea9727c4d88d1d8, server=jenkins-hbase4.apache.org,41759,1676444032818 in 271 msec 2023-02-15 06:53:59,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c2a14c04f55c99754ea9727c4d88d1d8, ASSIGN in 434 msec 2023-02-15 06:53:59,349 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] throttle.PressureAwareThroughputController(145): c2a14c04f55c99754ea9727c4d88d1d8#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:53:59,351 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] throttle.PressureAwareThroughputController(145): c2a14c04f55c99754ea9727c4d88d1d8#cf1#compaction#15 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:53:59,351 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924-top 2023-02-15 06:53:59,363 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49-top 2023-02-15 06:53:59,394 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37-top 2023-02-15 06:53:59,403 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58-top 2023-02-15 06:53:59,403 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/.tmp/cf1/f158c64172e04b0abce9b5d680bc60e1 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/f158c64172e04b0abce9b5d680bc60e1 2023-02-15 06:53:59,404 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/.tmp/cf2/db71b3c4e9274e49a4b6c9e9a3c6b06e as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/db71b3c4e9274e49a4b6c9e9a3c6b06e 2023-02-15 06:53:59,414 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-21c5747a134142ae90b63eb374381c53 2023-02-15 06:53:59,419 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-68ae222995854654a4551bf9752f2cf4 2023-02-15 06:53:59,423 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in c2a14c04f55c99754ea9727c4d88d1d8/cf2 of c2a14c04f55c99754ea9727c4d88d1d8 into db71b3c4e9274e49a4b6c9e9a3c6b06e(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:53:59,423 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in c2a14c04f55c99754ea9727c4d88d1d8/cf1 of c2a14c04f55c99754ea9727c4d88d1d8 into f158c64172e04b0abce9b5d680bc60e1(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:53:59,424 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c2a14c04f55c99754ea9727c4d88d1d8: 2023-02-15 06:53:59,424 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c2a14c04f55c99754ea9727c4d88d1d8: 2023-02-15 06:53:59,425 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8., storeName=c2a14c04f55c99754ea9727c4d88d1d8/cf1, priority=12, startTime=1676444039314; duration=0sec 2023-02-15 06:53:59,425 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8., storeName=c2a14c04f55c99754ea9727c4d88d1d8/cf2, priority=12, startTime=1676444039315; duration=0sec 2023-02-15 06:53:59,425 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-eae719281a7b435cb97500bfdc1e9fb7 2023-02-15 06:53:59,425 INFO [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(310): Store=a670097e89a3ff97d31f6ef5bcbe626a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:59,425 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:53:59,425 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:53:59,425 INFO [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:59,426 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2 2023-02-15 06:53:59,426 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2 2023-02-15 06:53:59,427 INFO [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a670097e89a3ff97d31f6ef5bcbe626a columnFamilyName cf2 2023-02-15 06:53:59,442 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5-top 2023-02-15 06:53:59,447 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052-top 2023-02-15 06:53:59,453 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161-top 2023-02-15 06:53:59,458 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c-top 2023-02-15 06:53:59,463 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-6c2de12cfabf497c95cabd50b8b8a777 2023-02-15 06:53:59,467 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-7ea249fb5ee4470bb66f9d13f8bb9022 2023-02-15 06:53:59,471 DEBUG [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-83d9ab44f2fa429fa4da6eb1e9952c5a 2023-02-15 06:53:59,471 INFO [StoreOpener-a670097e89a3ff97d31f6ef5bcbe626a-1] regionserver.HStore(310): Store=a670097e89a3ff97d31f6ef5bcbe626a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:53:59,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:59,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:59,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:53:59,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:53:59,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a670097e89a3ff97d31f6ef5bcbe626a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74403834, jitterRate=0.10870352387428284}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:53:59,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a670097e89a3ff97d31f6ef5bcbe626a: 2023-02-15 06:53:59,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a., pid=18, masterSystemTime=1676444039206 2023-02-15 06:53:59,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:53:59,481 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:53:59,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:53:59,481 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:53:59,481 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:53:59,481 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1912): a670097e89a3ff97d31f6ef5bcbe626a/cf1 is initiating minor compaction (all files) 2023-02-15 06:53:59,481 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:53:59,481 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1912): a670097e89a3ff97d31f6ef5bcbe626a/cf2 is initiating minor compaction (all files) 2023-02-15 06:53:59,481 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a670097e89a3ff97d31f6ef5bcbe626a/cf1 in testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:53:59,481 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a670097e89a3ff97d31f6ef5bcbe626a/cf2 in testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:53:59,482 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-68ae222995854654a4551bf9752f2cf4, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-21c5747a134142ae90b63eb374381c53, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-eae719281a7b435cb97500bfdc1e9fb7] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/.tmp, totalSize=34.9 K 2023-02-15 06:53:59,482 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-6c2de12cfabf497c95cabd50b8b8a777, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-7ea249fb5ee4470bb66f9d13f8bb9022, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-83d9ab44f2fa429fa4da6eb1e9952c5a] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/.tmp, totalSize=34.9 K 2023-02-15 06:53:59,483 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676444036181 2023-02-15 06:53:59,484 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676444036181 2023-02-15 06:53:59,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:53:59,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:53:59,485 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676444036634 2023-02-15 06:53:59,486 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676444036634 2023-02-15 06:53:59,486 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=a670097e89a3ff97d31f6ef5bcbe626a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:59,486 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676444036877 2023-02-15 06:53:59,486 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676444039486"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444039486"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444039486"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444039486"}]},"ts":"1676444039486"} 2023-02-15 06:53:59,487 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676444036877 2023-02-15 06:53:59,487 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676444037115 2023-02-15 06:53:59,488 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676444037115 2023-02-15 06:53:59,489 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-68ae222995854654a4551bf9752f2cf4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444037383 2023-02-15 06:53:59,489 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-6c2de12cfabf497c95cabd50b8b8a777, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444037383 2023-02-15 06:53:59,490 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-21c5747a134142ae90b63eb374381c53, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444037638 2023-02-15 06:53:59,490 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-eae719281a7b435cb97500bfdc1e9fb7, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444037871 2023-02-15 06:53:59,490 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-7ea249fb5ee4470bb66f9d13f8bb9022, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444037638 2023-02-15 06:53:59,491 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-83d9ab44f2fa429fa4da6eb1e9952c5a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444037871 2023-02-15 06:53:59,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-15 06:53:59,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure a670097e89a3ff97d31f6ef5bcbe626a, server=jenkins-hbase4.apache.org,41759,1676444032818 in 429 msec 2023-02-15 06:53:59,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-15 06:53:59,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a670097e89a3ff97d31f6ef5bcbe626a, ASSIGN in 599 msec 2023-02-15 06:53:59,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a in 1.6080 sec 2023-02-15 06:53:59,522 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] throttle.PressureAwareThroughputController(145): a670097e89a3ff97d31f6ef5bcbe626a#cf1#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:53:59,530 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] throttle.PressureAwareThroughputController(145): a670097e89a3ff97d31f6ef5bcbe626a#cf2#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:53:59,539 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/.tmp/cf1/f5f39087ba934b2881efb8ea97b19631 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/f5f39087ba934b2881efb8ea97b19631 2023-02-15 06:53:59,545 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/.tmp/cf2/a6019d8cda0b436db26519ac073ce59e as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/a6019d8cda0b436db26519ac073ce59e 2023-02-15 06:53:59,545 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a670097e89a3ff97d31f6ef5bcbe626a/cf1 of a670097e89a3ff97d31f6ef5bcbe626a into f5f39087ba934b2881efb8ea97b19631(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:53:59,546 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a670097e89a3ff97d31f6ef5bcbe626a: 2023-02-15 06:53:59,547 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a., storeName=a670097e89a3ff97d31f6ef5bcbe626a/cf1, priority=9, startTime=1676444039480; duration=0sec 2023-02-15 06:53:59,547 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:53:59,552 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a670097e89a3ff97d31f6ef5bcbe626a/cf2 of a670097e89a3ff97d31f6ef5bcbe626a into a6019d8cda0b436db26519ac073ce59e(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:53:59,553 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a670097e89a3ff97d31f6ef5bcbe626a: 2023-02-15 06:53:59,553 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a., storeName=a670097e89a3ff97d31f6ef5bcbe626a/cf2, priority=9, startTime=1676444039481; duration=0sec 2023-02-15 06:53:59,553 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:53:59,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41759] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:42146 deadline: 1676444099650, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776. is not online on jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:59,766 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-15 06:53:59,780 INFO [Listener at localhost/37669] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-15 06:53:59,783 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-15 06:53:59,816 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:53:59,827 INFO [Listener at localhost/37669] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-15 06:53:59,833 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-15 06:53:59,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:53:59,846 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444039845"}]},"ts":"1676444039845"} 2023-02-15 06:53:59,847 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-15 06:53:59,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-15 06:53:59,884 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-15 06:53:59,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c2a14c04f55c99754ea9727c4d88d1d8, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a670097e89a3ff97d31f6ef5bcbe626a, UNASSIGN}] 2023-02-15 06:53:59,888 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a670097e89a3ff97d31f6ef5bcbe626a, UNASSIGN 2023-02-15 06:53:59,889 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=a670097e89a3ff97d31f6ef5bcbe626a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:59,889 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676444039889"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444039889"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444039889"}]},"ts":"1676444039889"} 2023-02-15 06:53:59,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure a670097e89a3ff97d31f6ef5bcbe626a, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:53:59,895 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c2a14c04f55c99754ea9727c4d88d1d8, UNASSIGN 2023-02-15 06:53:59,896 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=c2a14c04f55c99754ea9727c4d88d1d8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:53:59,896 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676444039896"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444039896"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444039896"}]},"ts":"1676444039896"} 2023-02-15 06:53:59,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure c2a14c04f55c99754ea9727c4d88d1d8, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:53:59,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-15 06:54:00,044 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:00,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a670097e89a3ff97d31f6ef5bcbe626a, disabling compactions & flushes 2023-02-15 06:54:00,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:54:00,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:54:00,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. after waiting 0 ms 2023-02-15 06:54:00,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:54:00,056 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-68ae222995854654a4551bf9752f2cf4, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-21c5747a134142ae90b63eb374381c53, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-eae719281a7b435cb97500bfdc1e9fb7] to archive 2023-02-15 06:54:00,058 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:00,065 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776 2023-02-15 06:54:00,067 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776 2023-02-15 06:54:00,069 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776 2023-02-15 06:54:00,071 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776 2023-02-15 06:54:00,074 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-68ae222995854654a4551bf9752f2cf4 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-68ae222995854654a4551bf9752f2cf4 2023-02-15 06:54:00,076 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-21c5747a134142ae90b63eb374381c53 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-21c5747a134142ae90b63eb374381c53 2023-02-15 06:54:00,078 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-eae719281a7b435cb97500bfdc1e9fb7 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-eae719281a7b435cb97500bfdc1e9fb7 2023-02-15 06:54:00,097 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-6c2de12cfabf497c95cabd50b8b8a777, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-7ea249fb5ee4470bb66f9d13f8bb9022, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-83d9ab44f2fa429fa4da6eb1e9952c5a] to archive 2023-02-15 06:54:00,098 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:00,100 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776 2023-02-15 06:54:00,101 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776 2023-02-15 06:54:00,103 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776 2023-02-15 06:54:00,104 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776 2023-02-15 06:54:00,106 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-6c2de12cfabf497c95cabd50b8b8a777 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-6c2de12cfabf497c95cabd50b8b8a777 2023-02-15 06:54:00,108 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-7ea249fb5ee4470bb66f9d13f8bb9022 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-7ea249fb5ee4470bb66f9d13f8bb9022 2023-02-15 06:54:00,109 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-83d9ab44f2fa429fa4da6eb1e9952c5a to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/testSplitTableRegionUnevenDaughter=e93940bab783f139204747313e792776-83d9ab44f2fa429fa4da6eb1e9952c5a 2023-02-15 06:54:00,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-15 06:54:00,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:00,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a. 2023-02-15 06:54:00,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a670097e89a3ff97d31f6ef5bcbe626a: 2023-02-15 06:54:00,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:00,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:54:00,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c2a14c04f55c99754ea9727c4d88d1d8, disabling compactions & flushes 2023-02-15 06:54:00,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:54:00,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:54:00,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. after waiting 0 ms 2023-02-15 06:54:00,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:54:00,120 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=a670097e89a3ff97d31f6ef5bcbe626a, regionState=CLOSED 2023-02-15 06:54:00,121 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924-bottom] to archive 2023-02-15 06:54:00,121 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676444040120"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444040120"}]},"ts":"1676444040120"} 2023-02-15 06:54:00,123 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:00,126 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-15 06:54:00,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-15 06:54:00,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure a670097e89a3ff97d31f6ef5bcbe626a, server=jenkins-hbase4.apache.org,41759,1676444032818 in 234 msec 2023-02-15 06:54:00,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a670097e89a3ff97d31f6ef5bcbe626a, UNASSIGN in 243 msec 2023-02-15 06:54:00,132 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58.e93940bab783f139204747313e792776 2023-02-15 06:54:00,134 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37.e93940bab783f139204747313e792776 2023-02-15 06:54:00,136 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/833ac1bdcea04bce9637c68a0d427b49.e93940bab783f139204747313e792776 2023-02-15 06:54:00,138 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/70482e4c98e14aaf9ec682064c496924.e93940bab783f139204747313e792776 2023-02-15 06:54:00,140 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052-bottom] to archive 2023-02-15 06:54:00,142 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:00,145 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/48ac10727b6144f0bf91e9c72cba9ab5.e93940bab783f139204747313e792776 2023-02-15 06:54:00,147 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/cea384b791b14b5baadc4badcb91a84c.e93940bab783f139204747313e792776 2023-02-15 06:54:00,149 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/bb691aacb0a94e62be83e5cd26097161.e93940bab783f139204747313e792776 2023-02-15 06:54:00,150 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/a402ec297b3b4a808833bae0c78df052.e93940bab783f139204747313e792776 2023-02-15 06:54:00,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-15 06:54:00,159 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:00,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-15 06:54:00,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8. 2023-02-15 06:54:00,159 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c2a14c04f55c99754ea9727c4d88d1d8: 2023-02-15 06:54:00,161 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:54:00,162 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=c2a14c04f55c99754ea9727c4d88d1d8, regionState=CLOSED 2023-02-15 06:54:00,162 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676444040162"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444040162"}]},"ts":"1676444040162"} 2023-02-15 06:54:00,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-15 06:54:00,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure c2a14c04f55c99754ea9727c4d88d1d8, server=jenkins-hbase4.apache.org,41759,1676444032818 in 266 msec 2023-02-15 06:54:00,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-15 06:54:00,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c2a14c04f55c99754ea9727c4d88d1d8, UNASSIGN in 283 msec 2023-02-15 06:54:00,172 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444040172"}]},"ts":"1676444040172"} 2023-02-15 06:54:00,174 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-15 06:54:00,253 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-15 06:54:00,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 421 msec 2023-02-15 06:54:00,325 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-15 06:54:00,326 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-15 06:54:00,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-15 06:54:00,461 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-15 06:54:00,465 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-15 06:54:00,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:00,474 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:00,476 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:00,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-15 06:54:00,484 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776 2023-02-15 06:54:00,484 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:54:00,484 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:00,487 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/recovered.edits] 2023-02-15 06:54:00,487 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/recovered.edits] 2023-02-15 06:54:00,487 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/recovered.edits] 2023-02-15 06:54:00,492 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/21c5747a134142ae90b63eb374381c53 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/21c5747a134142ae90b63eb374381c53 2023-02-15 06:54:00,492 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/f5f39087ba934b2881efb8ea97b19631 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf1/f5f39087ba934b2881efb8ea97b19631 2023-02-15 06:54:00,493 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/f158c64172e04b0abce9b5d680bc60e1 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf1/f158c64172e04b0abce9b5d680bc60e1 2023-02-15 06:54:00,494 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/68ae222995854654a4551bf9752f2cf4 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/68ae222995854654a4551bf9752f2cf4 2023-02-15 06:54:00,496 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/70482e4c98e14aaf9ec682064c496924 2023-02-15 06:54:00,496 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/a6019d8cda0b436db26519ac073ce59e to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/cf2/a6019d8cda0b436db26519ac073ce59e 2023-02-15 06:54:00,496 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/db71b3c4e9274e49a4b6c9e9a3c6b06e to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/cf2/db71b3c4e9274e49a4b6c9e9a3c6b06e 2023-02-15 06:54:00,498 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/833ac1bdcea04bce9637c68a0d427b49 2023-02-15 06:54:00,500 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/a6d8ff142c9e45b799b5ddb3b4f1fa37 2023-02-15 06:54:00,500 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/recovered.edits/90.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a/recovered.edits/90.seqid 2023-02-15 06:54:00,501 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/recovered.edits/90.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8/recovered.edits/90.seqid 2023-02-15 06:54:00,501 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:00,502 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/c2a14c04f55c99754ea9727c4d88d1d8 2023-02-15 06:54:00,504 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/dc521b44c0ba44a1a8d9ba8d9c101f58 2023-02-15 06:54:00,505 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/eae719281a7b435cb97500bfdc1e9fb7 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf1/eae719281a7b435cb97500bfdc1e9fb7 2023-02-15 06:54:00,508 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/48ac10727b6144f0bf91e9c72cba9ab5 2023-02-15 06:54:00,510 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/6c2de12cfabf497c95cabd50b8b8a777 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/6c2de12cfabf497c95cabd50b8b8a777 2023-02-15 06:54:00,512 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/7ea249fb5ee4470bb66f9d13f8bb9022 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/7ea249fb5ee4470bb66f9d13f8bb9022 2023-02-15 06:54:00,513 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/83d9ab44f2fa429fa4da6eb1e9952c5a to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/83d9ab44f2fa429fa4da6eb1e9952c5a 2023-02-15 06:54:00,515 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/a402ec297b3b4a808833bae0c78df052 2023-02-15 06:54:00,516 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/bb691aacb0a94e62be83e5cd26097161 2023-02-15 06:54:00,518 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/cf2/cea384b791b14b5baadc4badcb91a84c 2023-02-15 06:54:00,521 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/recovered.edits/85.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776/recovered.edits/85.seqid 2023-02-15 06:54:00,521 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionUnevenDaughter/e93940bab783f139204747313e792776 2023-02-15 06:54:00,521 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-15 06:54:00,524 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:00,550 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-15 06:54:00,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-15 06:54:00,600 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-15 06:54:00,602 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:00,602 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-15 06:54:00,602 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444040602"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:00,602 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444040602"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:00,602 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444040602"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:00,606 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-15 06:54:00,606 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e93940bab783f139204747313e792776, NAME => 'testSplitTableRegionUnevenDaughter,,1676444035673.e93940bab783f139204747313e792776.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c2a14c04f55c99754ea9727c4d88d1d8, NAME => 'testSplitTableRegionUnevenDaughter,,1676444037890.c2a14c04f55c99754ea9727c4d88d1d8.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => a670097e89a3ff97d31f6ef5bcbe626a, NAME => 'testSplitTableRegionUnevenDaughter,26,1676444037890.a670097e89a3ff97d31f6ef5bcbe626a.', STARTKEY => '26', ENDKEY => ''}] 2023-02-15 06:54:00,606 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-15 06:54:00,606 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676444040606"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:00,610 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-15 06:54:00,677 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:00,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 212 msec 2023-02-15 06:54:00,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-15 06:54:00,781 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-15 06:54:00,805 INFO [Listener at localhost/37669] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=389 (was 358) Potentially hanging thread: hconnection-0x4c388f12-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:36368 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c388f12-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:35542 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:35886 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:36124 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:35896 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:46802 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:41759-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x4c388f12-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:35884 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c388f12-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:35836 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:46770 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4c388f12-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:36010 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:46782 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:49312 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:35902 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:35668 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:35848 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:35994 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=863 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=282 (was 245) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=4071 (was 3360) - AvailableMemoryMB LEAK? - 2023-02-15 06:54:00,821 INFO [Listener at localhost/37669] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=389, OpenFileDescriptor=863, MaxFileDescriptor=60000, SystemLoadAverage=282, ProcessCount=168, AvailableMemoryMB=4067 2023-02-15 06:54:00,821 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:00,831 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-15 06:54:00,831 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-15 06:54:00,833 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:00,835 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:54:00,836 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:54:00,838 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921 2023-02-15 06:54:00,838 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921 empty. 2023-02-15 06:54:00,839 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921 2023-02-15 06:54:00,839 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-15 06:54:00,859 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-15 06:54:00,860 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4593e78b81534378124b8d0b83524921, NAME => 'testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:54:00,870 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:00,871 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 4593e78b81534378124b8d0b83524921, disabling compactions & flushes 2023-02-15 06:54:00,871 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:00,871 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:00,871 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. after waiting 0 ms 2023-02-15 06:54:00,871 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:00,871 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:00,871 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 4593e78b81534378124b8d0b83524921: 2023-02-15 06:54:00,874 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:54:00,875 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444040875"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444040875"}]},"ts":"1676444040875"} 2023-02-15 06:54:00,877 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:54:00,883 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:54:00,884 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444040883"}]},"ts":"1676444040883"} 2023-02-15 06:54:00,885 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-15 06:54:00,889 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:54:00,890 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:54:00,890 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:54:00,890 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:54:00,890 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:54:00,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, ASSIGN}] 2023-02-15 06:54:00,893 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, ASSIGN 2023-02-15 06:54:00,893 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:54:01,044 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:54:01,045 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=4593e78b81534378124b8d0b83524921, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,045 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444041045"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444041045"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444041045"}]},"ts":"1676444041045"} 2023-02-15 06:54:01,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:54:01,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:01,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4593e78b81534378124b8d0b83524921, NAME => 'testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:54:01,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:01,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:01,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:01,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:01,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:01,206 INFO [StoreOpener-4593e78b81534378124b8d0b83524921-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:01,208 DEBUG [StoreOpener-4593e78b81534378124b8d0b83524921-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1 2023-02-15 06:54:01,208 DEBUG [StoreOpener-4593e78b81534378124b8d0b83524921-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1 2023-02-15 06:54:01,209 INFO [StoreOpener-4593e78b81534378124b8d0b83524921-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4593e78b81534378124b8d0b83524921 columnFamilyName cf1 2023-02-15 06:54:01,209 INFO [StoreOpener-4593e78b81534378124b8d0b83524921-1] regionserver.HStore(310): Store=4593e78b81534378124b8d0b83524921/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:01,209 INFO [StoreOpener-4593e78b81534378124b8d0b83524921-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:01,211 DEBUG [StoreOpener-4593e78b81534378124b8d0b83524921-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2 2023-02-15 06:54:01,212 DEBUG [StoreOpener-4593e78b81534378124b8d0b83524921-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2 2023-02-15 06:54:01,213 INFO [StoreOpener-4593e78b81534378124b8d0b83524921-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4593e78b81534378124b8d0b83524921 columnFamilyName cf2 2023-02-15 06:54:01,214 INFO [StoreOpener-4593e78b81534378124b8d0b83524921-1] regionserver.HStore(310): Store=4593e78b81534378124b8d0b83524921/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:01,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921 2023-02-15 06:54:01,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921 2023-02-15 06:54:01,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:01,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:01,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:54:01,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4593e78b81534378124b8d0b83524921; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61921328, jitterRate=-0.0773003101348877}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:01,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4593e78b81534378124b8d0b83524921: 2023-02-15 06:54:01,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921., pid=27, masterSystemTime=1676444041200 2023-02-15 06:54:01,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:01,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:01,226 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=4593e78b81534378124b8d0b83524921, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444041226"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444041226"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444041226"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444041226"}]},"ts":"1676444041226"} 2023-02-15 06:54:01,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-15 06:54:01,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 in 180 msec 2023-02-15 06:54:01,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-15 06:54:01,234 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, ASSIGN in 341 msec 2023-02-15 06:54:01,234 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:54:01,235 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444041234"}]},"ts":"1676444041234"} 2023-02-15 06:54:01,236 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-15 06:54:01,240 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:54:01,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 408 msec 2023-02-15 06:54:01,345 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:01,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-15 06:54:01,348 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-15 06:54:01,348 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,348 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-15 06:54:01,349 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-15 06:54:01,349 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,349 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,350 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,350 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,350 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,350 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,351 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,351 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,352 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:01,353 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,352 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:01,353 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:01,353 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:01,353 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:01,353 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:01,353 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:01,354 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:01,354 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,356 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:01,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:01,356 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:01,356 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,356 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,357 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,357 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:01,361 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:01,361 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:01,361 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:01,361 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:01,361 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. started... 2023-02-15 06:54:01,361 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:01,361 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:01,364 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4593e78b81534378124b8d0b83524921 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-15 06:54:01,364 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,365 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,365 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:01,376 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/a010a042d8a64a289e7a49807028e658 2023-02-15 06:54:01,399 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73 2023-02-15 06:54:01,405 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/a010a042d8a64a289e7a49807028e658 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658 2023-02-15 06:54:01,411 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:54:01,412 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73 2023-02-15 06:54:01,417 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:54:01,418 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4593e78b81534378124b8d0b83524921 in 54ms, sequenceid=6, compaction requested=false 2023-02-15 06:54:01,418 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-15 06:54:01,419 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4593e78b81534378124b8d0b83524921: 2023-02-15 06:54:01,419 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:01,419 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:01,419 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:01,419 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,419 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:01,419 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:01,421 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,421 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:01,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:01,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:01,422 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,422 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:01,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:01,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:01,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-15 06:54:01,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c2bd1a0[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:01,424 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-15 06:54:01,425 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,426 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,426 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,426 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,426 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,426 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:01,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,426 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:01,426 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:01,426 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:01,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,426 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:01,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,426 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:01,426 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:01,427 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:01,427 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:01,427 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:01,428 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,428 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:01,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:01,428 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,428 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:01,428 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:01,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:01,429 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,429 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:01,430 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:01,430 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,430 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:01,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:01,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:01,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:01,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:54:01,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-15 06:54:01,435 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-15 06:54:01,435 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:01,435 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:01,435 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-15 06:54:01,435 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:01,437 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,437 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,437 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:01,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:01,437 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,437 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,438 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,438 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:01,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,439 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:01,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,439 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,440 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:01,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,442 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,442 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,442 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,445 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-15 06:54:01,445 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:01,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,445 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,446 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,445 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,447 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:01,446 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,447 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:01,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,447 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,447 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,547 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:01,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:01,571 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:01,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:01,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-15 06:54:01,573 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-15 06:54:01,573 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,574 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-15 06:54:01,574 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-15 06:54:01,575 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,575 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,576 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,576 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,577 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,576 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,577 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,578 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:01,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:01,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,578 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:01,579 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:01,579 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:01,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:01,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,579 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:01,579 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:01,579 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:01,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,579 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:01,580 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:01,587 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:01,580 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:01,587 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:01,587 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:01,587 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:01,587 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:01,587 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,587 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:01,587 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,587 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:01,589 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:01,589 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:01,588 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. started... 2023-02-15 06:54:01,588 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:01,589 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:01,589 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:01,589 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:01,589 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4593e78b81534378124b8d0b83524921 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:01,589 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:01,591 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,593 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,593 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:01,593 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,594 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,594 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:01,613 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/236308abe9df47a19053f4c0e7f12c30 2023-02-15 06:54:01,646 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/1e3052e03365461c8a2b146aeac3809b 2023-02-15 06:54:01,652 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/236308abe9df47a19053f4c0e7f12c30 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30 2023-02-15 06:54:01,658 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:54:01,659 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/1e3052e03365461c8a2b146aeac3809b as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b 2023-02-15 06:54:01,664 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:54:01,666 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4593e78b81534378124b8d0b83524921 in 77ms, sequenceid=19, compaction requested=false 2023-02-15 06:54:01,666 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4593e78b81534378124b8d0b83524921: 2023-02-15 06:54:01,666 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:01,666 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:01,666 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:01,666 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,666 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:01,666 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:01,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,668 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:01,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:01,668 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,668 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:01,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:01,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-15 06:54:01,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47ffb623[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:01,671 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-15 06:54:01,671 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,672 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,672 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,672 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,673 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:01,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,673 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:01,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,673 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,673 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:01,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,673 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:01,673 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:01,673 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:01,673 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:01,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,674 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:01,674 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:01,674 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:01,675 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:01,675 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,675 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:01,679 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:01,679 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,680 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:01,681 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:01,681 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,681 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:01,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:01,682 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:01,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:01,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:01,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:01,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:54:01,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-15 06:54:01,692 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-15 06:54:01,692 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:01,692 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:01,692 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-15 06:54:01,692 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:01,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:01,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:01,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:01,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,696 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,697 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,697 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:01,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:01,700 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,701 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,701 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,707 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-15 06:54:01,707 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:01,707 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,708 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:01,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,713 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,713 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,713 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,713 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,737 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-15 06:54:01,738 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-15 06:54:01,738 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:01,738 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-15 06:54:01,813 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:01,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:01,835 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:01,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:01,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-15 06:54:01,837 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-15 06:54:01,837 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,838 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-15 06:54:01,838 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-15 06:54:01,838 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,839 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,840 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,840 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,840 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,840 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,841 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,841 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:01,841 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,841 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,841 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:01,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:01,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:01,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,842 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:01,842 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:01,842 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:01,842 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:01,842 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:01,842 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:01,842 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:01,842 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:01,842 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,842 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:01,842 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:01,843 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:01,843 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:01,844 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,843 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:01,844 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:01,843 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:01,845 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:01,845 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:01,845 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:01,844 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:01,845 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:01,845 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:01,845 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:01,845 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:01,846 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. started... 2023-02-15 06:54:01,846 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4593e78b81534378124b8d0b83524921 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:01,847 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,847 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,848 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,848 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:01,848 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,848 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:01,863 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/ed33e3638efb408c9307d76815feb817 2023-02-15 06:54:01,895 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/7416fd5ad8ea49828d3f94abf501c8bd 2023-02-15 06:54:01,901 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/ed33e3638efb408c9307d76815feb817 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817 2023-02-15 06:54:01,911 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:54:01,912 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/7416fd5ad8ea49828d3f94abf501c8bd as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd 2023-02-15 06:54:01,918 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:54:01,919 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4593e78b81534378124b8d0b83524921 in 73ms, sequenceid=32, compaction requested=true 2023-02-15 06:54:01,919 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4593e78b81534378124b8d0b83524921: 2023-02-15 06:54:01,919 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:01,919 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:01,919 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:01,919 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,919 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:01,919 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:01,921 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,921 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:01,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:01,921 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,921 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:01,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:01,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:01,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:01,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-15 06:54:01,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4530b1a0[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:01,923 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-15 06:54:01,923 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,926 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,926 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,926 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,926 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,926 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:01,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,926 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:01,926 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:01,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,927 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:01,927 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:01,927 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:01,927 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:01,927 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:01,927 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:01,927 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:01,928 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:01,928 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:01,928 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,928 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,928 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:01,928 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,930 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:01,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:01,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:01,930 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:01,931 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,931 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:01,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:01,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:01,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:01,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:54:01,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-15 06:54:01,934 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-15 06:54:01,935 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:01,935 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:01,935 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-15 06:54:01,935 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:01,936 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,936 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,936 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,936 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,937 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:01,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:01,939 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,939 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:01,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,940 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,940 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:01,941 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,943 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:01,943 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:01,943 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,948 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,948 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,948 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,948 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,948 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,948 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-15 06:54:01,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,948 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:01,948 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:01,948 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:01,949 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:01,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:01,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:01,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:01,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:01,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:01,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,050 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:02,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:02,077 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:02,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:02,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-15 06:54:02,079 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-15 06:54:02,079 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,080 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-15 06:54:02,080 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-15 06:54:02,081 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,081 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,082 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,082 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,082 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,082 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,082 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,083 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:02,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:02,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,084 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:02,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:02,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,084 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:02,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:02,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:02,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:02,090 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:02,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:02,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:02,090 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:02,090 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,090 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:02,090 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:02,091 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:02,091 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:02,091 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:02,091 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:02,091 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,092 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:02,092 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:02,092 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:02,094 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:02,094 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:02,094 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:02,094 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:02,095 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. started... 2023-02-15 06:54:02,095 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4593e78b81534378124b8d0b83524921 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:02,098 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,099 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,099 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,099 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:02,099 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,099 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:02,119 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/a86f32767df448a7bb8f4d0aa191f081 2023-02-15 06:54:02,147 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/684057df8ad8473eb0b3e2d9cf33ded2 2023-02-15 06:54:02,155 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/a86f32767df448a7bb8f4d0aa191f081 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081 2023-02-15 06:54:02,161 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:54:02,162 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/684057df8ad8473eb0b3e2d9cf33ded2 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2 2023-02-15 06:54:02,167 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:54:02,169 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4593e78b81534378124b8d0b83524921 in 74ms, sequenceid=45, compaction requested=true 2023-02-15 06:54:02,169 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4593e78b81534378124b8d0b83524921: 2023-02-15 06:54:02,169 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:02,169 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:02,169 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:02,169 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,169 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:02,169 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:02,171 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:02,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:02,171 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:02,172 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,172 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:02,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:02,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-15 06:54:02,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1cceccec[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:02,174 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-15 06:54:02,174 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,177 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,177 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,177 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,177 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:02,177 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:02,177 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:02,177 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:02,177 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:02,177 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:02,177 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,177 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:02,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,178 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:02,178 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:02,178 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:02,179 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:02,179 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,179 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:02,181 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:02,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:02,181 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:02,181 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,181 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:02,181 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:02,182 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,182 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:02,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:02,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:02,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:02,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:54:02,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-15 06:54:02,190 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-15 06:54:02,190 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:02,190 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:02,190 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-15 06:54:02,190 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:02,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,193 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,193 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:02,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:02,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,193 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:02,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,194 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:02,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,196 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,196 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:02,197 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,207 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,206 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,207 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,208 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,208 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,208 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,208 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,208 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-15 06:54:02,208 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:02,208 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:02,209 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:02,309 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:02,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:02,336 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:02,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:02,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-15 06:54:02,339 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-15 06:54:02,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-15 06:54:02,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-15 06:54:02,340 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,340 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,341 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,341 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,341 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,341 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,342 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,342 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:02,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:02,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:02,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:02,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:02,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:02,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:02,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:02,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:02,344 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:02,345 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:02,345 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:02,345 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:02,345 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. started... 2023-02-15 06:54:02,345 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4593e78b81534378124b8d0b83524921 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:02,345 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,347 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,347 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,347 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:02,348 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,348 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:02,392 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/251cfefbc0d145548b2cbded1f3027ad 2023-02-15 06:54:02,421 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/707949499a864c709bde841d4f15089c 2023-02-15 06:54:02,428 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/251cfefbc0d145548b2cbded1f3027ad as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad 2023-02-15 06:54:02,433 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:54:02,434 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/707949499a864c709bde841d4f15089c as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c 2023-02-15 06:54:02,449 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:54:02,451 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4593e78b81534378124b8d0b83524921 in 105ms, sequenceid=58, compaction requested=true 2023-02-15 06:54:02,451 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4593e78b81534378124b8d0b83524921: 2023-02-15 06:54:02,451 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:02,454 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:02,454 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:02,454 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,454 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:02,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-15 06:54:02,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2edab804[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:02,456 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-15 06:54:02,456 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,458 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,458 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,458 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:02,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,458 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,458 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:02,460 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,460 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,460 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:02,460 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:02,460 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,460 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:02,460 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:02,461 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:02,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:02,461 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,461 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:02,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:02,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:54:02,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-15 06:54:02,464 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-15 06:54:02,464 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:02,464 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:02,464 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-15 06:54:02,464 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:02,467 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,467 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,467 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,468 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:02,469 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,469 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:02,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,471 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,471 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,471 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,476 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,477 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-15 06:54:02,477 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:02,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,478 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,478 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,478 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,478 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:02,477 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,477 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,478 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:02,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,478 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,578 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:02,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:02,601 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:02,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:02,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-15 06:54:02,603 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-15 06:54:02,603 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,603 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-15 06:54:02,603 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-15 06:54:02,604 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,604 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,605 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,605 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,605 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,605 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,607 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,607 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:02,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,607 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:02,608 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:02,608 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-15 06:54:02,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:02,608 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:02,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:02,608 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,608 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:02,608 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,608 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:02,608 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:02,608 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:02,608 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-15 06:54:02,608 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:02,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-15 06:54:02,608 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-15 06:54:02,608 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:02,609 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-15 06:54:02,609 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:02,609 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:02,609 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:02,609 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:02,609 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:02,609 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. started... 2023-02-15 06:54:02,609 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4593e78b81534378124b8d0b83524921 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:02,610 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,610 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,611 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,611 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:02,611 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,611 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:02,628 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/dad583dd59e94d7f88dc2b90afd5b14e 2023-02-15 06:54:02,652 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/dc45294790624833905e1cefc1ae12cc 2023-02-15 06:54:02,658 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/dad583dd59e94d7f88dc2b90afd5b14e as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e 2023-02-15 06:54:02,665 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:54:02,666 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/dc45294790624833905e1cefc1ae12cc as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc 2023-02-15 06:54:02,671 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:54:02,673 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4593e78b81534378124b8d0b83524921 in 64ms, sequenceid=71, compaction requested=true 2023-02-15 06:54:02,673 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4593e78b81534378124b8d0b83524921: 2023-02-15 06:54:02,673 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:02,673 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:02,673 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:02,673 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,673 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-15 06:54:02,673 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-15 06:54:02,675 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,675 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:02,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:02,676 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,676 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:02,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:02,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:02,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:02,678 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-15 06:54:02,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14ae4d52[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:02,679 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-15 06:54:02,679 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,681 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,681 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,681 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,682 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:02,682 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,682 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,682 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:02,682 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:02,682 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:02,682 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:02,682 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-15 06:54:02,682 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-15 06:54:02,682 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:02,682 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:02,684 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:02,684 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,684 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:02,685 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:02,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-15 06:54:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:02,685 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-15 06:54:02,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:02,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:02,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:02,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:54:02,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-15 06:54:02,690 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-15 06:54:02,690 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:02,690 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:02,690 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-15 06:54:02,690 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:02,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,692 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,693 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,694 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,696 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:02,696 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-15 06:54:02,697 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,701 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,701 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,701 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,701 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,701 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:02,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,701 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,701 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,701 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,701 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-15 06:54:02,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:02,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:02,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,701 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:02,701 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:02,702 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:02,703 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:02,702 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,703 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:02,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:02,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-15 06:54:02,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:02,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:02,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:02,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:02,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-15 06:54:02,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-15 06:54:02,803 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:02,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:03,072 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-15 06:54:03,073 DEBUG [Listener at localhost/37669] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:03,075 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,076 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,076 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,076 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,077 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:03,077 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:03,078 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:03,328 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,330 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:03,330 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:03,330 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:03,333 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-15 06:54:03,334 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:03,334 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:03,334 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:03,334 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:03,335 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:03,339 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-15 06:54:03,341 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:03,342 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:03,342 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:03,343 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:03,343 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:03,344 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:03,344 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:03,345 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:03,352 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:03,353 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,353 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:03,353 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,353 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,353 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,354 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 18 msec 2023-02-15 06:54:03,360 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:03,361 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:03,366 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444073366 2023-02-15 06:54:03,366 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:03,371 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,377 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:03,377 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=4593e78b81534378124b8d0b83524921, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41759,1676444032818, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818, openSeqNum=2 2023-02-15 06:54:03,377 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:03,378 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444103378 2023-02-15 06:54:03,378 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444163378 2023-02-15 06:54:03,378 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 11 msec 2023-02-15 06:54:03,378 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:03,383 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:03,383 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:03,383 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:03,628 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,628 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:03,628 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:03,628 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:03,629 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:03,629 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:03,629 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:03,629 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:03,629 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:03,630 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:03,632 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-15 06:54:03,633 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:03,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:03,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:03,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:03,635 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:03,636 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:03,636 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:03,636 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:03,637 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:03,637 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,638 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:03,638 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,638 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,638 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,638 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-15 06:54:03,639 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:03,640 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:03,644 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444073644 2023-02-15 06:54:03,645 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:03,645 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,648 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:03,649 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=4593e78b81534378124b8d0b83524921, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41759,1676444032818, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818, openSeqNum=2 2023-02-15 06:54:03,649 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:03,649 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444103649 2023-02-15 06:54:03,649 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444163649 2023-02-15 06:54:03,649 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:03,649 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:03,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN}] 2023-02-15 06:54:03,652 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:03,652 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:03,652 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:03,899 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,900 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:03,900 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:03,900 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:03,901 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:03,901 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:03,901 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:03,901 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:03,901 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:03,902 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:03,904 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-15 06:54:03,904 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:03,905 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:03,905 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:03,905 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:03,906 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:03,906 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:03,906 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:03,906 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:03,907 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:03,907 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,907 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:03,907 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,907 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,908 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,908 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-15 06:54:03,908 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:03,909 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:03,909 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444073909 2023-02-15 06:54:03,909 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:03,912 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:03,914 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:03,914 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=4593e78b81534378124b8d0b83524921, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41759,1676444032818, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818, openSeqNum=2 2023-02-15 06:54:03,915 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:03,915 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444103915 2023-02-15 06:54:03,915 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444163915 2023-02-15 06:54:03,915 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-15 06:54:03,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN}] 2023-02-15 06:54:03,915 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:03,918 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:03,919 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4593e78b81534378124b8d0b83524921, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:03,919 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444043919"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444043919"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444043919"}]},"ts":"1676444043919"} 2023-02-15 06:54:03,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:54:03,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:03,921 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:03,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:03,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:04,165 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,165 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:04,165 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:04,165 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:04,166 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:04,167 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:04,167 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:04,167 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:04,167 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:04,167 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:04,169 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-15 06:54:04,169 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-15 06:54:04,169 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:04,170 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:04,170 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:04,171 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:04,171 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,172 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:04,172 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,172 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,173 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:04,174 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,174 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:04,174 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:04,174 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,174 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,175 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,175 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:04,175 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:04,175 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-15 06:54:04,176 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:04,176 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:04,176 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921 to restore RIT 2023-02-15 06:54:04,177 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444074177 2023-02-15 06:54:04,178 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:04,178 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:04,181 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:04,182 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=4593e78b81534378124b8d0b83524921, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,41759,1676444032818, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818, openSeqNum=2 2023-02-15 06:54:04,182 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:04,182 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444104182 2023-02-15 06:54:04,182 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444164182 2023-02-15 06:54:04,182 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-15 06:54:04,182 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4593e78b81534378124b8d0b83524921, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:04,183 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444044182"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444044182"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444044182"}]},"ts":"1676444044182"} 2023-02-15 06:54:04,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:54:04,185 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:04,338 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4593e78b81534378124b8d0b83524921, disabling compactions & flushes 2023-02-15 06:54:04,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:04,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:04,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. after waiting 0 ms 2023-02-15 06:54:04,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:04,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 4593e78b81534378124b8d0b83524921 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-15 06:54:04,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/ebfadab6bb3f419bb2ff36f383f61c53 2023-02-15 06:54:04,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/a1afddb21a1046709757145cf1bfa4b3 2023-02-15 06:54:04,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf1/ebfadab6bb3f419bb2ff36f383f61c53 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53 2023-02-15 06:54:04,382 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:54:04,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/.tmp/cf2/a1afddb21a1046709757145cf1bfa4b3 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3 2023-02-15 06:54:04,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:54:04,390 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4593e78b81534378124b8d0b83524921 in 51ms, sequenceid=82, compaction requested=true 2023-02-15 06:54:04,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-15 06:54:04,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:04,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. 2023-02-15 06:54:04,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4593e78b81534378124b8d0b83524921: 2023-02-15 06:54:04,422 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4593e78b81534378124b8d0b83524921, regionState=CLOSED 2023-02-15 06:54:04,423 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444044422"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444044422"}]},"ts":"1676444044422"} 2023-02-15 06:54:04,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,425 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:04,425 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:04,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:04,425 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:04,432 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,432 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:04,432 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:04,432 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:04,433 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:04,433 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:04,433 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:04,433 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:04,433 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:04,434 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:04,436 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-15 06:54:04,436 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-15 06:54:04,436 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-15 06:54:04,436 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:04,437 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:04,437 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:04,438 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:04,438 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,439 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:04,439 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,439 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,440 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:04,441 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,441 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:04,441 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:04,441 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:04,442 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,442 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,442 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,442 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:04,442 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:04,442 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:04,442 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-15 06:54:04,443 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:04,444 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:04,444 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921 to restore RIT 2023-02-15 06:54:04,444 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444074444 2023-02-15 06:54:04,445 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:04,445 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:04,448 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:04,448 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=4593e78b81534378124b8d0b83524921, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41759,1676444032818, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818, openSeqNum=2 2023-02-15 06:54:04,448 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:04,448 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444104448 2023-02-15 06:54:04,448 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444164448 2023-02-15 06:54:04,448 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:04,449 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4593e78b81534378124b8d0b83524921, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:04,449 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444044448"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444044448"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444044448"}]},"ts":"1676444044448"} 2023-02-15 06:54:04,451 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:04,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-15 06:54:04,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 in 266 msec 2023-02-15 06:54:04,463 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:04,463 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:04,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:04,463 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:04,699 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,699 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:04,699 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:04,699 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:04,699 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:04,699 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:04,699 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:04,699 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:04,699 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:04,700 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:04,702 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-15 06:54:04,702 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-15 06:54:04,702 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:04,702 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:04,703 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:04,703 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:04,703 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:04,704 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,704 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:04,705 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,705 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,706 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:04,707 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,707 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:04,707 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:04,707 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:04,707 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,707 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,707 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,708 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:04,708 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:04,708 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-15 06:54:04,709 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:04,709 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:04,709 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921 to restore RIT 2023-02-15 06:54:04,710 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444074710 2023-02-15 06:54:04,711 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:04,711 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:04,714 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:04,715 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=4593e78b81534378124b8d0b83524921, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41759,1676444032818, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818, openSeqNum=2 2023-02-15 06:54:04,715 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:04,715 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444104715 2023-02-15 06:54:04,715 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444164715 2023-02-15 06:54:04,715 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-15 06:54:04,715 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:04,718 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:04,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-15 06:54:04,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN in 803 msec 2023-02-15 06:54:04,724 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:04,724 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:04,724 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:04,965 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,966 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:04,966 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:04,966 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:04,967 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:04,967 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:04,967 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:04,967 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:04,967 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:04,968 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:04,970 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-15 06:54:04,970 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:04,970 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:04,970 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:04,971 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:04,971 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:04,971 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:04,972 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,973 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:04,973 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,973 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:04,974 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:04,974 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,975 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:04,975 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:04,975 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:04,975 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:04,975 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:04,975 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-15 06:54:04,976 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:04,976 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:04,977 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444074977 2023-02-15 06:54:04,978 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,978 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:04,981 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:04,981 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=4593e78b81534378124b8d0b83524921, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41759,1676444032818, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818, openSeqNum=2 2023-02-15 06:54:04,982 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:04,982 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444104982 2023-02-15 06:54:04,982 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444164982 2023-02-15 06:54:04,982 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:04,982 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,982 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,982 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:04,987 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:04,997 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=4593e78b81534378124b8d0b83524921, threads=14 2023-02-15 06:54:04,998 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,998 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,998 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,998 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,999 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,999 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,999 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,999 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,999 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:04,999 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,000 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,005 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,005 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,005 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,194 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,194 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,199 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,205 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,214 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,228 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,256 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,256 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,256 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,260 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,260 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,261 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,263 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,263 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,263 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 4593e78b81534378124b8d0b83524921 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-15 06:54:05,307 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:05,307 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:05,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:05,309 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:05,482 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:05,482 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:05,482 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:05,482 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:05,483 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:05,484 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:05,484 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:05,484 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:05,484 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:05,485 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:05,486 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-15 06:54:05,486 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:05,486 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:05,486 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:05,487 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:05,487 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:05,487 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:05,488 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:05,488 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:05,489 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:05,489 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:05,489 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:05,490 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:05,490 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:05,490 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:05,490 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:05,490 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:05,490 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:05,491 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:05,491 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:05,491 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:05,491 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-15 06:54:05,491 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:05,492 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:05,492 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444075492 2023-02-15 06:54:05,494 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:05,494 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:05,496 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:05,497 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=4593e78b81534378124b8d0b83524921, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41759,1676444032818, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818, openSeqNum=2 2023-02-15 06:54:05,497 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:05,497 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444105497 2023-02-15 06:54:05,497 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444165497 2023-02-15 06:54:05,497 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-15 06:54:05,508 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=4593e78b81534378124b8d0b83524921, threads=14 2023-02-15 06:54:05,513 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,513 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,513 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,513 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,513 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,513 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,516 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,516 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,516 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,520 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,520 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,520 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,520 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,521 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,620 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-15 06:54:05,758 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,781 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,788 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,792 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,812 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,816 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,832 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,837 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,837 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,838 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,844 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,848 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,852 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,853 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30 for region: 4593e78b81534378124b8d0b83524921 2023-02-15 06:54:05,853 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 4593e78b81534378124b8d0b83524921 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-15 06:54:05,897 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:05,902 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:54:05,904 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:54:05,904 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:05,904 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:05,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:05,904 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:05,997 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:05,998 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:05,998 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:05,998 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:05,999 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:05,999 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:05,999 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:05,999 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:05,999 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:06,000 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:06,001 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-15 06:54:06,001 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:06,001 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:06,002 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:06,002 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:06,002 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:06,002 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:06,003 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:06,003 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:06,003 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:06,003 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:06,004 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:06,004 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,004 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:06,004 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:06,005 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:06,005 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,005 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,005 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,005 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:06,005 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:06,005 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-15 06:54:06,006 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:06,006 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:06,007 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444076007 2023-02-15 06:54:06,009 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:06,009 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,011 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:06,012 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=4593e78b81534378124b8d0b83524921, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41759,1676444032818, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818, openSeqNum=2 2023-02-15 06:54:06,012 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:06,012 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444106012 2023-02-15 06:54:06,012 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444166012 2023-02-15 06:54:06,012 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-15 06:54:06,015 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:06,017 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:06,017 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:06,017 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:06,262 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:06,262 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:06,262 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:06,262 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:06,263 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:06,263 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:06,263 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:06,263 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:06,263 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:06,264 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:06,265 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-15 06:54:06,265 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:06,265 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:06,265 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:06,265 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:06,265 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:06,266 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:06,266 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:06,266 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:06,266 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:06,267 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:06,267 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:06,267 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,267 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN 2023-02-15 06:54:06,268 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:06,268 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:06,268 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,268 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,268 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,268 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4593e78b81534378124b8d0b83524921, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:06,268 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 4593e78b81534378124b8d0b83524921, server=jenkins-hbase4.apache.org,41759,1676444032818 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:06,268 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-15 06:54:06,269 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:06,269 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:06,269 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444076269 2023-02-15 06:54:06,270 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:06,271 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,273 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:06,273 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=4593e78b81534378124b8d0b83524921, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41759,1676444032818, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818, openSeqNum=2 2023-02-15 06:54:06,273 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:06,273 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444106273 2023-02-15 06:54:06,273 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444166273 2023-02-15 06:54:06,274 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:06,277 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444046277"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676444046277"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676444046277"}]},"ts":"1676444046277"} 2023-02-15 06:54:06,277 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444046277"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444046277"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444046277"}]},"ts":"1676444046277"} 2023-02-15 06:54:06,277 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444046277"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444046277"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444046277"}]},"ts":"1676444046277"} 2023-02-15 06:54:06,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9163391b02da2883a80eb51ae90b7f1c, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b14b2480de7d04e57425d9cf4365200f, ASSIGN}] 2023-02-15 06:54:06,284 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b14b2480de7d04e57425d9cf4365200f, ASSIGN 2023-02-15 06:54:06,285 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b14b2480de7d04e57425d9cf4365200f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:54:06,286 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9163391b02da2883a80eb51ae90b7f1c, ASSIGN 2023-02-15 06:54:06,287 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9163391b02da2883a80eb51ae90b7f1c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:54:06,435 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-15 06:54:06,437 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=b14b2480de7d04e57425d9cf4365200f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:06,438 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444046437"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444046437"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444046437"}]},"ts":"1676444046437"} 2023-02-15 06:54:06,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure b14b2480de7d04e57425d9cf4365200f, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:54:06,444 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=9163391b02da2883a80eb51ae90b7f1c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:06,444 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444046443"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444046443"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444046443"}]},"ts":"1676444046443"} 2023-02-15 06:54:06,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 9163391b02da2883a80eb51ae90b7f1c, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:54:06,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:06,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b14b2480de7d04e57425d9cf4365200f, NAME => 'testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.', STARTKEY => '41', ENDKEY => ''} 2023-02-15 06:54:06,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:06,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:06,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,598 INFO [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,599 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1 2023-02-15 06:54:06,599 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1 2023-02-15 06:54:06,600 INFO [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b14b2480de7d04e57425d9cf4365200f columnFamilyName cf1 2023-02-15 06:54:06,615 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30-top 2023-02-15 06:54:06,619 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad-top 2023-02-15 06:54:06,623 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658-top 2023-02-15 06:54:06,627 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081-top 2023-02-15 06:54:06,632 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e-top 2023-02-15 06:54:06,637 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53-top 2023-02-15 06:54:06,641 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817-top 2023-02-15 06:54:06,642 INFO [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(310): Store=b14b2480de7d04e57425d9cf4365200f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:06,642 INFO [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,643 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2 2023-02-15 06:54:06,643 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2 2023-02-15 06:54:06,643 INFO [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b14b2480de7d04e57425d9cf4365200f columnFamilyName cf2 2023-02-15 06:54:06,658 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b-top 2023-02-15 06:54:06,663 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73-top 2023-02-15 06:54:06,667 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2-top 2023-02-15 06:54:06,671 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c-top 2023-02-15 06:54:06,675 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd-top 2023-02-15 06:54:06,680 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3-top 2023-02-15 06:54:06,684 DEBUG [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc-top 2023-02-15 06:54:06,685 INFO [StoreOpener-b14b2480de7d04e57425d9cf4365200f-1] regionserver.HStore(310): Store=b14b2480de7d04e57425d9cf4365200f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:06,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:06,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:06,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b14b2480de7d04e57425d9cf4365200f; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59319020, jitterRate=-0.116077721118927}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:06,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b14b2480de7d04e57425d9cf4365200f: 2023-02-15 06:54:06,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f., pid=33, masterSystemTime=1676444046592 2023-02-15 06:54:06,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:06,693 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:54:06,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:54:06,693 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:54:06,693 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:06,693 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1912): b14b2480de7d04e57425d9cf4365200f/cf1 is initiating minor compaction (all files) 2023-02-15 06:54:06,693 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:06,693 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b14b2480de7d04e57425d9cf4365200f/cf1 in testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:06,693 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1912): b14b2480de7d04e57425d9cf4365200f/cf2 is initiating minor compaction (all files) 2023-02-15 06:54:06,693 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b14b2480de7d04e57425d9cf4365200f/cf2 in testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:06,693 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53-top] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/.tmp, totalSize=34.9 K 2023-02-15 06:54:06,694 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3-top] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/.tmp, totalSize=34.9 K 2023-02-15 06:54:06,694 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676444041338 2023-02-15 06:54:06,694 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676444041338 2023-02-15 06:54:06,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:06,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:06,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:06,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9163391b02da2883a80eb51ae90b7f1c, NAME => 'testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.', STARTKEY => '', ENDKEY => '41'} 2023-02-15 06:54:06,695 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676444041550 2023-02-15 06:54:06,695 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676444041550 2023-02-15 06:54:06,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:06,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:06,696 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676444041816 2023-02-15 06:54:06,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:06,696 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=b14b2480de7d04e57425d9cf4365200f, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:06,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:06,696 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676444041816 2023-02-15 06:54:06,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:06,696 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444046696"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444046696"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444046696"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444046696"}]},"ts":"1676444046696"} 2023-02-15 06:54:06,696 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676444042053 2023-02-15 06:54:06,697 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676444042053 2023-02-15 06:54:06,697 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676444042312 2023-02-15 06:54:06,698 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676444042312 2023-02-15 06:54:06,698 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676444042581 2023-02-15 06:54:06,698 INFO [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:06,698 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676444042806 2023-02-15 06:54:06,698 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676444042581 2023-02-15 06:54:06,699 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676444042806 2023-02-15 06:54:06,701 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1 2023-02-15 06:54:06,701 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1 2023-02-15 06:54:06,702 INFO [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9163391b02da2883a80eb51ae90b7f1c columnFamilyName cf1 2023-02-15 06:54:06,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-15 06:54:06,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure b14b2480de7d04e57425d9cf4365200f, server=jenkins-hbase4.apache.org,41759,1676444032818 in 258 msec 2023-02-15 06:54:06,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b14b2480de7d04e57425d9cf4365200f, ASSIGN in 421 msec 2023-02-15 06:54:06,721 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] throttle.PressureAwareThroughputController(145): b14b2480de7d04e57425d9cf4365200f#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:06,729 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] throttle.PressureAwareThroughputController(145): b14b2480de7d04e57425d9cf4365200f#cf2#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:06,733 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30-bottom 2023-02-15 06:54:06,738 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad-bottom 2023-02-15 06:54:06,755 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/.tmp/cf1/78bfc72cabbe414fb788def933b067af as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/78bfc72cabbe414fb788def933b067af 2023-02-15 06:54:06,757 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658-bottom 2023-02-15 06:54:06,761 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b14b2480de7d04e57425d9cf4365200f/cf1 of b14b2480de7d04e57425d9cf4365200f into 78bfc72cabbe414fb788def933b067af(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:06,761 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b14b2480de7d04e57425d9cf4365200f: 2023-02-15 06:54:06,761 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f., storeName=b14b2480de7d04e57425d9cf4365200f/cf1, priority=9, startTime=1676444046692; duration=0sec 2023-02-15 06:54:06,761 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:06,765 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081-bottom 2023-02-15 06:54:06,769 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e-bottom 2023-02-15 06:54:06,773 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/.tmp/cf2/1fa992c21d10451391e9b4d8a7a38694 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/1fa992c21d10451391e9b4d8a7a38694 2023-02-15 06:54:06,775 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53-bottom 2023-02-15 06:54:06,779 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817-bottom 2023-02-15 06:54:06,779 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b14b2480de7d04e57425d9cf4365200f/cf2 of b14b2480de7d04e57425d9cf4365200f into 1fa992c21d10451391e9b4d8a7a38694(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:06,779 INFO [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(310): Store=9163391b02da2883a80eb51ae90b7f1c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:06,780 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b14b2480de7d04e57425d9cf4365200f: 2023-02-15 06:54:06,780 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f., storeName=b14b2480de7d04e57425d9cf4365200f/cf2, priority=9, startTime=1676444046693; duration=0sec 2023-02-15 06:54:06,780 INFO [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:06,780 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:06,781 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2 2023-02-15 06:54:06,781 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2 2023-02-15 06:54:06,781 INFO [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9163391b02da2883a80eb51ae90b7f1c columnFamilyName cf2 2023-02-15 06:54:06,797 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b-bottom 2023-02-15 06:54:06,801 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73-bottom 2023-02-15 06:54:06,805 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2-bottom 2023-02-15 06:54:06,809 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c-bottom 2023-02-15 06:54:06,813 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd-bottom 2023-02-15 06:54:06,818 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3-bottom 2023-02-15 06:54:06,822 DEBUG [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc-bottom 2023-02-15 06:54:06,822 INFO [StoreOpener-9163391b02da2883a80eb51ae90b7f1c-1] regionserver.HStore(310): Store=9163391b02da2883a80eb51ae90b7f1c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:06,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:06,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:06,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:06,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:06,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9163391b02da2883a80eb51ae90b7f1c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60909098, jitterRate=-0.09238371253013611}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:06,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9163391b02da2883a80eb51ae90b7f1c: 2023-02-15 06:54:06,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c., pid=34, masterSystemTime=1676444046592 2023-02-15 06:54:06,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:06,830 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:54:06,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:54:06,830 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:54:06,830 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:06,830 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1912): 9163391b02da2883a80eb51ae90b7f1c/cf1 is initiating minor compaction (all files) 2023-02-15 06:54:06,830 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:06,830 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9163391b02da2883a80eb51ae90b7f1c/cf1 in testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:06,830 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1912): 9163391b02da2883a80eb51ae90b7f1c/cf2 is initiating minor compaction (all files) 2023-02-15 06:54:06,830 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 9163391b02da2883a80eb51ae90b7f1c/cf2 in testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:06,831 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53-bottom] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/.tmp, totalSize=34.9 K 2023-02-15 06:54:06,831 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3-bottom] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/.tmp, totalSize=34.9 K 2023-02-15 06:54:06,832 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444041338 2023-02-15 06:54:06,832 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444041338 2023-02-15 06:54:06,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:06,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:06,832 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444041550 2023-02-15 06:54:06,833 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=9163391b02da2883a80eb51ae90b7f1c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:06,833 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444041550 2023-02-15 06:54:06,833 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444041816 2023-02-15 06:54:06,833 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444046833"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444046833"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444046833"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444046833"}]},"ts":"1676444046833"} 2023-02-15 06:54:06,833 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444041816 2023-02-15 06:54:06,834 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444042053 2023-02-15 06:54:06,834 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444042053 2023-02-15 06:54:06,834 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444042312 2023-02-15 06:54:06,835 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444042312 2023-02-15 06:54:06,835 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444042581 2023-02-15 06:54:06,835 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444042581 2023-02-15 06:54:06,835 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444042806 2023-02-15 06:54:06,835 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444042806 2023-02-15 06:54:06,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-15 06:54:06,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 9163391b02da2883a80eb51ae90b7f1c, server=jenkins-hbase4.apache.org,41759,1676444032818 in 390 msec 2023-02-15 06:54:06,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-15 06:54:06,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9163391b02da2883a80eb51ae90b7f1c, ASSIGN in 556 msec 2023-02-15 06:54:06,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f in 3.7680 sec 2023-02-15 06:54:06,852 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] throttle.PressureAwareThroughputController(145): 9163391b02da2883a80eb51ae90b7f1c#cf2#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:06,854 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9163391b02da2883a80eb51ae90b7f1c#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:06,872 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/.tmp/cf1/10c9f7dd8bcf4b039237579bde216b66 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/10c9f7dd8bcf4b039237579bde216b66 2023-02-15 06:54:06,872 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/.tmp/cf2/b14cc41a482b4f3fa6e7e3bbd647421f as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/b14cc41a482b4f3fa6e7e3bbd647421f 2023-02-15 06:54:06,879 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9163391b02da2883a80eb51ae90b7f1c/cf1 of 9163391b02da2883a80eb51ae90b7f1c into 10c9f7dd8bcf4b039237579bde216b66(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:06,879 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9163391b02da2883a80eb51ae90b7f1c: 2023-02-15 06:54:06,879 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c., storeName=9163391b02da2883a80eb51ae90b7f1c/cf1, priority=9, startTime=1676444046829; duration=0sec 2023-02-15 06:54:06,879 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:06,880 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9163391b02da2883a80eb51ae90b7f1c/cf2 of 9163391b02da2883a80eb51ae90b7f1c into b14cc41a482b4f3fa6e7e3bbd647421f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:06,880 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9163391b02da2883a80eb51ae90b7f1c: 2023-02-15 06:54:06,880 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c., storeName=9163391b02da2883a80eb51ae90b7f1c/cf2, priority=9, startTime=1676444046830; duration=0sec 2023-02-15 06:54:06,880 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:07,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41759] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:42146 deadline: 1676444107025, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921. is not online on jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:07,138 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-15 06:54:07,147 INFO [Listener at localhost/37669] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-15 06:54:07,149 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-15 06:54:07,169 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:07,171 INFO [Listener at localhost/37669] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-15 06:54:07,171 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-15 06:54:07,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:07,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-15 06:54:07,176 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444047176"}]},"ts":"1676444047176"} 2023-02-15 06:54:07,177 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-15 06:54:07,179 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-15 06:54:07,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9163391b02da2883a80eb51ae90b7f1c, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b14b2480de7d04e57425d9cf4365200f, UNASSIGN}] 2023-02-15 06:54:07,181 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b14b2480de7d04e57425d9cf4365200f, UNASSIGN 2023-02-15 06:54:07,181 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=b14b2480de7d04e57425d9cf4365200f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:07,181 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444047181"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444047181"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444047181"}]},"ts":"1676444047181"} 2023-02-15 06:54:07,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure b14b2480de7d04e57425d9cf4365200f, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:54:07,185 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9163391b02da2883a80eb51ae90b7f1c, UNASSIGN 2023-02-15 06:54:07,186 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=9163391b02da2883a80eb51ae90b7f1c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:07,186 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444047186"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444047186"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444047186"}]},"ts":"1676444047186"} 2023-02-15 06:54:07,187 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 9163391b02da2883a80eb51ae90b7f1c, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:54:07,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-15 06:54:07,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:07,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b14b2480de7d04e57425d9cf4365200f, disabling compactions & flushes 2023-02-15 06:54:07,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:07,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:07,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. after waiting 0 ms 2023-02-15 06:54:07,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:07,347 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53-top] to archive 2023-02-15 06:54:07,348 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:07,350 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,352 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,353 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,354 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,355 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,357 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,358 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,359 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3-top] to archive 2023-02-15 06:54:07,360 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:07,362 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,363 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,365 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,366 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,367 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,368 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,369 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-15 06:54:07,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:07,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f. 2023-02-15 06:54:07,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b14b2480de7d04e57425d9cf4365200f: 2023-02-15 06:54:07,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:07,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:07,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9163391b02da2883a80eb51ae90b7f1c, disabling compactions & flushes 2023-02-15 06:54:07,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:07,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:07,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. after waiting 0 ms 2023-02-15 06:54:07,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:07,376 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=b14b2480de7d04e57425d9cf4365200f, regionState=CLOSED 2023-02-15 06:54:07,376 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444047376"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444047376"}]},"ts":"1676444047376"} 2023-02-15 06:54:07,378 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53-bottom] to archive 2023-02-15 06:54:07,379 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:07,381 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-15 06:54:07,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure b14b2480de7d04e57425d9cf4365200f, server=jenkins-hbase4.apache.org,41759,1676444032818 in 195 msec 2023-02-15 06:54:07,383 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b14b2480de7d04e57425d9cf4365200f, UNASSIGN in 201 msec 2023-02-15 06:54:07,384 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,386 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,387 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,389 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,390 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,392 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3-bottom] to archive 2023-02-15 06:54:07,392 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:07,394 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,395 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,396 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,397 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,398 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,399 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,401 DEBUG [StoreCloser-testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-15 06:54:07,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:07,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c. 2023-02-15 06:54:07,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9163391b02da2883a80eb51ae90b7f1c: 2023-02-15 06:54:07,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:07,408 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=9163391b02da2883a80eb51ae90b7f1c, regionState=CLOSED 2023-02-15 06:54:07,408 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444047408"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444047408"}]},"ts":"1676444047408"} 2023-02-15 06:54:07,411 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-15 06:54:07,411 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 9163391b02da2883a80eb51ae90b7f1c, server=jenkins-hbase4.apache.org,41759,1676444032818 in 222 msec 2023-02-15 06:54:07,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-15 06:54:07,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=9163391b02da2883a80eb51ae90b7f1c, UNASSIGN in 231 msec 2023-02-15 06:54:07,414 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444047414"}]},"ts":"1676444047414"} 2023-02-15 06:54:07,416 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-15 06:54:07,418 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-15 06:54:07,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 247 msec 2023-02-15 06:54:07,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-15 06:54:07,478 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-15 06:54:07,479 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-15 06:54:07,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:07,482 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:07,483 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:07,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-15 06:54:07,490 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,490 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:07,490 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:07,493 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/recovered.edits] 2023-02-15 06:54:07,493 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/recovered.edits] 2023-02-15 06:54:07,493 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/recovered.edits] 2023-02-15 06:54:07,497 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/.regioninfo to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/.regioninfo 2023-02-15 06:54:07,497 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/.regioninfo to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/.regioninfo 2023-02-15 06:54:07,497 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/236308abe9df47a19053f4c0e7f12c30 2023-02-15 06:54:07,499 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/251cfefbc0d145548b2cbded1f3027ad 2023-02-15 06:54:07,501 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a010a042d8a64a289e7a49807028e658 2023-02-15 06:54:07,501 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,501 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/236308abe9df47a19053f4c0e7f12c30.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,502 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/a86f32767df448a7bb8f4d0aa191f081 2023-02-15 06:54:07,502 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,502 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/251cfefbc0d145548b2cbded1f3027ad.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,504 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/dad583dd59e94d7f88dc2b90afd5b14e 2023-02-15 06:54:07,504 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,504 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/a010a042d8a64a289e7a49807028e658.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,505 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ebfadab6bb3f419bb2ff36f383f61c53 2023-02-15 06:54:07,506 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,506 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/a86f32767df448a7bb8f4d0aa191f081.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,507 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf1/ed33e3638efb408c9307d76815feb817 2023-02-15 06:54:07,508 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,508 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/dad583dd59e94d7f88dc2b90afd5b14e.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,509 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,510 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/ebfadab6bb3f419bb2ff36f383f61c53.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,511 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,511 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/1e3052e03365461c8a2b146aeac3809b 2023-02-15 06:54:07,512 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf1/ed33e3638efb408c9307d76815feb817.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,513 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73 2023-02-15 06:54:07,515 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,515 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/684057df8ad8473eb0b3e2d9cf33ded2 2023-02-15 06:54:07,517 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,517 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/707949499a864c709bde841d4f15089c 2023-02-15 06:54:07,518 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/1e3052e03365461c8a2b146aeac3809b.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,519 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/7416fd5ad8ea49828d3f94abf501c8bd 2023-02-15 06:54:07,519 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,520 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/4cfa49cfc01d4fcf8ed1f8aa5839df73.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,521 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,522 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/a1afddb21a1046709757145cf1bfa4b3 2023-02-15 06:54:07,522 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/684057df8ad8473eb0b3e2d9cf33ded2.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,523 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,524 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/707949499a864c709bde841d4f15089c.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,524 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/cf2/dc45294790624833905e1cefc1ae12cc 2023-02-15 06:54:07,525 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,526 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/7416fd5ad8ea49828d3f94abf501c8bd.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,528 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/b14b2480de7d04e57425d9cf4365200f/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,528 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/a1afddb21a1046709757145cf1bfa4b3.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,529 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/recovered.edits/85.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921/recovered.edits/85.seqid 2023-02-15 06:54:07,530 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/9163391b02da2883a80eb51ae90b7f1c/cf2/dc45294790624833905e1cefc1ae12cc.4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,530 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/4593e78b81534378124b8d0b83524921 2023-02-15 06:54:07,531 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/78bfc72cabbe414fb788def933b067af to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf1/78bfc72cabbe414fb788def933b067af 2023-02-15 06:54:07,533 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/10c9f7dd8bcf4b039237579bde216b66 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf1/10c9f7dd8bcf4b039237579bde216b66 2023-02-15 06:54:07,534 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/1fa992c21d10451391e9b4d8a7a38694 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/cf2/1fa992c21d10451391e9b4d8a7a38694 2023-02-15 06:54:07,536 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/b14cc41a482b4f3fa6e7e3bbd647421f to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/cf2/b14cc41a482b4f3fa6e7e3bbd647421f 2023-02-15 06:54:07,538 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/recovered.edits/90.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f/recovered.edits/90.seqid 2023-02-15 06:54:07,539 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:07,540 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/recovered.edits/90.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c/recovered.edits/90.seqid 2023-02-15 06:54:07,541 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitWithoutPONR/9163391b02da2883a80eb51ae90b7f1c 2023-02-15 06:54:07,541 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-15 06:54:07,543 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:07,545 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-15 06:54:07,548 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-15 06:54:07,549 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:07,549 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-15 06:54:07,549 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444047549"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:07,549 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444047549"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:07,550 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444047549"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:07,551 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-15 06:54:07,552 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4593e78b81534378124b8d0b83524921, NAME => 'testSplitWithoutPONR,,1676444040832.4593e78b81534378124b8d0b83524921.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 9163391b02da2883a80eb51ae90b7f1c, NAME => 'testSplitWithoutPONR,,1676444043073.9163391b02da2883a80eb51ae90b7f1c.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => b14b2480de7d04e57425d9cf4365200f, NAME => 'testSplitWithoutPONR,41,1676444043073.b14b2480de7d04e57425d9cf4365200f.', STARTKEY => '41', ENDKEY => ''}] 2023-02-15 06:54:07,552 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-15 06:54:07,552 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676444047552"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:07,553 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-15 06:54:07,555 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:07,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 76 msec 2023-02-15 06:54:07,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-15 06:54:07,585 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-15 06:54:07,603 INFO [Listener at localhost/37669] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=395 (was 389) Potentially hanging thread: hconnection-0x4c388f12-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:36124 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:53470 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:53506 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:53490 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:40118 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:40102 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:53498 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:35884 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:53456 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:40084 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:40088 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:49230 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:36010 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:40126 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:49238 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1651163308_15 at /127.0.0.1:40090 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=859 (was 863), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=276 (was 282), ProcessCount=168 (was 168), AvailableMemoryMB=3278 (was 4067) 2023-02-15 06:54:07,620 INFO [Listener at localhost/37669] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=395, OpenFileDescriptor=859, MaxFileDescriptor=60000, SystemLoadAverage=276, ProcessCount=168, AvailableMemoryMB=3277 2023-02-15 06:54:07,621 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:07,624 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-15 06:54:07,624 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-15 06:54:07,626 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:07,628 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:54:07,629 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:54:07,630 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:07,631 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df empty. 2023-02-15 06:54:07,631 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:07,631 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-15 06:54:07,644 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-15 06:54:07,646 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => f6039dce9e5bf374c6575ade89f0e2df, NAME => 'testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:54:07,654 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:07,655 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing f6039dce9e5bf374c6575ade89f0e2df, disabling compactions & flushes 2023-02-15 06:54:07,655 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:07,655 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:07,655 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. after waiting 0 ms 2023-02-15 06:54:07,655 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:07,655 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:07,655 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:07,657 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:54:07,658 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444047658"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444047658"}]},"ts":"1676444047658"} 2023-02-15 06:54:07,660 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:54:07,660 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:54:07,661 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444047661"}]},"ts":"1676444047661"} 2023-02-15 06:54:07,662 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-15 06:54:07,665 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:54:07,667 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:54:07,667 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:54:07,667 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:54:07,667 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:54:07,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN}] 2023-02-15 06:54:07,669 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:07,669 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35089,1676444032716; forceNewPlan=false, retain=false 2023-02-15 06:54:07,819 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:54:07,821 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:07,821 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444047821"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444047821"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444047821"}]},"ts":"1676444047821"} 2023-02-15 06:54:07,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:54:07,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:07,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f6039dce9e5bf374c6575ade89f0e2df, NAME => 'testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:54:07,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:07,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:07,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:07,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:07,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:07,984 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:07,985 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1 2023-02-15 06:54:07,986 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1 2023-02-15 06:54:07,986 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f6039dce9e5bf374c6575ade89f0e2df columnFamilyName cf1 2023-02-15 06:54:07,987 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(310): Store=f6039dce9e5bf374c6575ade89f0e2df/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:07,987 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:07,988 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2 2023-02-15 06:54:07,988 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2 2023-02-15 06:54:07,989 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f6039dce9e5bf374c6575ade89f0e2df columnFamilyName cf2 2023-02-15 06:54:07,989 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(310): Store=f6039dce9e5bf374c6575ade89f0e2df/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:07,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:07,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:07,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:07,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:07,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:54:07,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f6039dce9e5bf374c6575ade89f0e2df; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61207062, jitterRate=-0.08794370293617249}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:07,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:07,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df., pid=43, masterSystemTime=1676444047974 2023-02-15 06:54:07,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:07,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:07,999 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:07,999 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444047999"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444047999"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444047999"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444047999"}]},"ts":"1676444047999"} 2023-02-15 06:54:08,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-15 06:54:08,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 in 177 msec 2023-02-15 06:54:08,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-15 06:54:08,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN in 335 msec 2023-02-15 06:54:08,005 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:54:08,006 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444048005"}]},"ts":"1676444048005"} 2023-02-15 06:54:08,007 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-15 06:54:08,013 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:54:08,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 388 msec 2023-02-15 06:54:08,129 DEBUG [Listener at localhost/37669] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-15 06:54:08,132 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56944, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-15 06:54:08,140 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:08,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-15 06:54:08,141 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-15 06:54:08,141 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,142 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-15 06:54:08,142 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:54:08,142 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,142 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,143 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,143 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,144 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,143 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,144 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,144 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,146 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,146 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,146 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,146 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,146 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,146 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,146 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,146 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:08,146 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:08,146 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,146 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. started... 2023-02-15 06:54:08,146 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,147 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f6039dce9e5bf374c6575ade89f0e2df 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-15 06:54:08,146 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:08,146 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,147 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,147 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,147 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,147 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,147 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,149 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,150 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,163 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/4ddff29601ed48c89f1d180c026b0143 2023-02-15 06:54:08,178 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/56a140bece9d457a873231c410fc1aa8 2023-02-15 06:54:08,183 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/4ddff29601ed48c89f1d180c026b0143 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143 2023-02-15 06:54:08,187 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:54:08,188 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/56a140bece9d457a873231c410fc1aa8 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8 2023-02-15 06:54:08,191 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:54:08,192 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for f6039dce9e5bf374c6575ade89f0e2df in 45ms, sequenceid=6, compaction requested=false 2023-02-15 06:54:08,192 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-15 06:54:08,193 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:08,193 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:08,193 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:08,193 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:08,193 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,193 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,193 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,196 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,196 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,196 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,196 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-15 06:54:08,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54d353d6[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:08,199 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:54:08,199 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,200 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,200 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,200 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,200 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,201 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,201 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:08,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,201 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,201 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,201 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:08,201 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,201 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,201 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,201 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:08,201 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,204 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,204 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:54:08,208 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-15 06:54:08,208 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:08,208 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:08,208 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-15 06:54:08,208 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:08,211 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,211 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,211 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,211 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,211 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,211 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,211 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,211 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,212 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,212 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,215 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,215 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,215 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,218 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,218 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,218 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,218 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,219 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,219 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,218 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,219 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-15 06:54:08,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,219 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,219 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,219 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,219 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,219 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:08,219 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,219 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,219 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,220 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,220 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:08,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,220 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:08,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,220 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,220 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,220 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,320 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:08,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:08,337 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:08,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:08,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-15 06:54:08,339 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-15 06:54:08,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-15 06:54:08,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:54:08,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,340 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,340 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,340 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,340 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,341 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,341 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,347 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,347 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,347 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,347 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,347 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,347 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:08,348 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:08,348 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. started... 2023-02-15 06:54:08,349 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f6039dce9e5bf374c6575ade89f0e2df 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:08,348 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,350 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,350 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,350 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,350 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,350 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,353 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,353 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,354 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,363 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/3c3102a38a0f48f481e35ee24e114482 2023-02-15 06:54:08,383 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/569a18e0ece64733b9d60678c869e827 2023-02-15 06:54:08,389 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/3c3102a38a0f48f481e35ee24e114482 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482 2023-02-15 06:54:08,394 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:54:08,394 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/569a18e0ece64733b9d60678c869e827 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827 2023-02-15 06:54:08,399 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:54:08,400 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f6039dce9e5bf374c6575ade89f0e2df in 52ms, sequenceid=19, compaction requested=false 2023-02-15 06:54:08,400 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:08,400 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:08,400 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:08,400 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:08,400 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,400 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,400 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,402 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,402 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,402 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,402 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-15 06:54:08,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e105c0c[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:08,404 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:54:08,404 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,405 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,405 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,405 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,406 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,406 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,406 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,406 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,406 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,406 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,406 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:08,406 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,406 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,406 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:08,406 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:08,407 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,408 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,408 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,412 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,412 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,412 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,412 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,412 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,412 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,412 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:54:08,415 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:54:08,415 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-15 06:54:08,415 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:08,415 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:08,415 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-15 06:54:08,415 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:08,417 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,417 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,418 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,420 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,421 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,422 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,422 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,425 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,425 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,425 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,425 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,425 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,425 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,425 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,425 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,425 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-15 06:54:08,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,425 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,426 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:08,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,425 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,425 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,426 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,426 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,426 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,426 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,426 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,426 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,426 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:08,426 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,426 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:08,527 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:08,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:08,544 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:08,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:08,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-15 06:54:08,545 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-15 06:54:08,545 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,545 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-15 06:54:08,545 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:54:08,546 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,546 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,547 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,547 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,547 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,547 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,548 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,548 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:08,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,550 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,549 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,550 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,550 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:08,550 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:08,550 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:08,550 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. started... 2023-02-15 06:54:08,550 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f6039dce9e5bf374c6575ade89f0e2df 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:08,553 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,553 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,554 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,554 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,554 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,554 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,564 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/9cd8800f408f46188974b73872a75d1d 2023-02-15 06:54:08,578 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/71da874e4fb8428185831cde722d7cea 2023-02-15 06:54:08,582 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/9cd8800f408f46188974b73872a75d1d as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d 2023-02-15 06:54:08,587 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:54:08,588 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/71da874e4fb8428185831cde722d7cea as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea 2023-02-15 06:54:08,593 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:54:08,593 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f6039dce9e5bf374c6575ade89f0e2df in 43ms, sequenceid=32, compaction requested=true 2023-02-15 06:54:08,594 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:08,594 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:08,594 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:08,594 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:08,594 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,594 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,594 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,595 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,596 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,596 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-15 06:54:08,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4571d952[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:08,598 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:54:08,598 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,599 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,599 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,599 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,599 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,599 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:08,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,600 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:08,600 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,600 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,600 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:08,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,600 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,600 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,600 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:08,601 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,601 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,601 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,603 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,603 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,603 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,603 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,603 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,603 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,603 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:54:08,607 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:54:08,607 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-15 06:54:08,607 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:08,607 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:08,607 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-15 06:54:08,607 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:08,609 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,609 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,609 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,609 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,609 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,609 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,609 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,612 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,613 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,613 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,615 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,616 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,616 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,621 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,621 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,621 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,621 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,621 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,621 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,621 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,621 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,621 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-15 06:54:08,622 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,621 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,622 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,622 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:08,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,622 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,622 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,622 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,622 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:08,622 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,623 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:08,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,623 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,623 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,623 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,723 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:08,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:08,740 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:08,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:08,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-15 06:54:08,742 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-15 06:54:08,742 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,742 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-15 06:54:08,742 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:54:08,743 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,743 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,745 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,745 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,745 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,745 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,746 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,746 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:08,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,747 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,747 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,747 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,747 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,747 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,748 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,747 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:08,748 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:08,748 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:08,748 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. started... 2023-02-15 06:54:08,749 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f6039dce9e5bf374c6575ade89f0e2df 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:08,750 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,750 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,750 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,750 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,750 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,750 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,759 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/af286c32355a410998b8d4944b538098 2023-02-15 06:54:08,773 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/1d49ba87102b4b8aae24c629324c945b 2023-02-15 06:54:08,778 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/af286c32355a410998b8d4944b538098 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098 2023-02-15 06:54:08,784 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:54:08,785 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/1d49ba87102b4b8aae24c629324c945b as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b 2023-02-15 06:54:08,790 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:54:08,791 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f6039dce9e5bf374c6575ade89f0e2df in 42ms, sequenceid=45, compaction requested=true 2023-02-15 06:54:08,791 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:08,792 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:08,792 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:08,792 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:08,792 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,792 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,792 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,793 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,793 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,794 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,794 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-15 06:54:08,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2353c873[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:08,796 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:54:08,796 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,797 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,797 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,797 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,798 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,798 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,798 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:08,798 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,798 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,798 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:08,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,798 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,798 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:08,798 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,798 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,798 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:08,801 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,801 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,801 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,801 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,801 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,801 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,802 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,802 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,802 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,802 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:54:08,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:54:08,805 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-15 06:54:08,805 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:08,805 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:08,805 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-15 06:54:08,805 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:08,807 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,807 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,807 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,807 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,807 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,807 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,808 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,808 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,809 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,809 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,811 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,811 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,811 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,815 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,815 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,815 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,815 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,815 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,815 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,815 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,815 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,815 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-15 06:54:08,815 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:08,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,815 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,815 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,816 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:08,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,816 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:08,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,816 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,816 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,816 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,816 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,816 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,816 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,816 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,816 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,916 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:08,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:08,932 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:08,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:08,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-15 06:54:08,934 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-15 06:54:08,934 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,934 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-15 06:54:08,934 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:54:08,935 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,935 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,937 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,937 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,937 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,937 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:08,938 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,938 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,939 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,939 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,939 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,939 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:08,940 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:08,940 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. started... 2023-02-15 06:54:08,940 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f6039dce9e5bf374c6575ade89f0e2df 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:08,941 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,942 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,942 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,942 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,942 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,942 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,950 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/530190ce10fb48db8924ad24178c76fa 2023-02-15 06:54:08,963 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/335613eacaf84668bd7e8a6d1f189222 2023-02-15 06:54:08,968 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/530190ce10fb48db8924ad24178c76fa as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa 2023-02-15 06:54:08,972 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:54:08,973 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/335613eacaf84668bd7e8a6d1f189222 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222 2023-02-15 06:54:08,977 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:54:08,978 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f6039dce9e5bf374c6575ade89f0e2df in 38ms, sequenceid=58, compaction requested=true 2023-02-15 06:54:08,978 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:08,978 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:08,978 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:08,978 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:08,978 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,978 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:08,979 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:08,980 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,980 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,980 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,981 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:08,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-15 06:54:08,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6dd0722c[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:08,982 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:54:08,983 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,985 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,985 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,985 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,985 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,985 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:08,985 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:08,985 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,985 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,985 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:08,985 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,986 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:08,986 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:08,986 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:08,985 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:08,987 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,987 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,987 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,987 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,991 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,991 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:08,991 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,992 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:08,992 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,992 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:08,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:08,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:54:08,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:54:08,994 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-15 06:54:08,994 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:08,995 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:08,995 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-15 06:54:08,995 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:08,997 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,997 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,997 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:08,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:08,997 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,997 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:08,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:08,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,998 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:08,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:08,998 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:08,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:08,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:09,000 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:09,000 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,001 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,003 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,003 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:09,003 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:09,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:09,004 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-15 06:54:09,004 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:09,004 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:09,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:09,004 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:09,003 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:09,003 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:09,004 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:09,004 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:09,003 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,004 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:09,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:09,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:09,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:09,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:09,004 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,005 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,005 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,005 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:09,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:09,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:09,005 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:09,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:09,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:09,005 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,005 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,005 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,005 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,104 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:09,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:09,120 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:09,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:09,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-15 06:54:09,122 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-15 06:54:09,122 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:09,122 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-15 06:54:09,122 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:54:09,123 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,123 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,124 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:09,124 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:09,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:09,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:09,124 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:09,124 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:09,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:09,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:09,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:09,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,125 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,125 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:09,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:09,126 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:09,126 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:09,126 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:09,126 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:09,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:09,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-15 06:54:09,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:09,127 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:54:09,127 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. started... 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:09,127 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f6039dce9e5bf374c6575ade89f0e2df 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:09,128 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,130 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,130 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,130 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:09,131 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,131 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:09,140 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/410fa16390d6437ab1f2a18cb643aebb 2023-02-15 06:54:09,153 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/1ffbf9cf284c446f981fb23311466f00 2023-02-15 06:54:09,158 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/410fa16390d6437ab1f2a18cb643aebb as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb 2023-02-15 06:54:09,162 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:54:09,163 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/1ffbf9cf284c446f981fb23311466f00 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00 2023-02-15 06:54:09,168 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:54:09,170 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f6039dce9e5bf374c6575ade89f0e2df in 43ms, sequenceid=71, compaction requested=true 2023-02-15 06:54:09,170 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:09,170 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:09,170 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:09,170 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:09,170 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,170 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-15 06:54:09,170 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-15 06:54:09,172 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,172 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:09,172 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,172 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:09,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:09,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:09,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:09,175 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-15 06:54:09,175 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@519cb3a6[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:09,175 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:54:09,175 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,176 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,176 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,176 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,176 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:09,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,176 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,176 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:09,176 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:09,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-15 06:54:09,177 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:09,178 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:09,178 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,178 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:09,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,178 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:09,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:09,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:09,178 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:09,178 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:09,179 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:09,181 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-15 06:54:09,181 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:09,181 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-15 06:54:09,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:09,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:54:09,185 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:54:09,185 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-15 06:54:09,185 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:09,185 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:09,185 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-15 06:54:09,185 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:09,187 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,187 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,187 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,187 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:09,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,187 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,187 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:09,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:09,188 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,187 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:09,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:09,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:09,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:09,188 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:09,189 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:09,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:09,191 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-15 06:54:09,192 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,192 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:09,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:09,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-15 06:54:09,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:09,197 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-15 06:54:09,198 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:09,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:09,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:09,197 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:09,197 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:09,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:09,198 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:09,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:09,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:09,198 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:09,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:09,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:09,198 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:09,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:09,198 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,199 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:09,199 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,199 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-15 06:54:09,199 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-15 06:54:09,299 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:09,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:09,562 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-15 06:54:09,564 DEBUG [Listener at localhost/37669] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:09,565 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,566 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,566 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,566 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,567 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:09,567 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:09,567 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:09,815 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,815 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:09,815 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:09,815 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:09,816 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-15 06:54:09,816 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:09,816 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:09,816 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:09,816 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:09,817 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:09,818 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-15 06:54:09,819 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:09,819 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:09,820 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:09,820 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:09,820 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:09,821 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:09,821 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:09,821 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:09,822 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:09,822 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:09,822 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:09,823 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:09,823 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:09,823 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,824 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:09,824 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,824 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,824 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,824 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-15 06:54:09,825 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:09,825 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:09,826 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444079826 2023-02-15 06:54:09,828 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:09,828 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,831 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:09,831 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:09,831 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:09,831 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444109831 2023-02-15 06:54:09,831 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444169831 2023-02-15 06:54:09,832 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:09,832 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:09,834 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:09,834 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:09,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:09,834 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:10,082 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,082 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:10,082 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:10,082 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:10,083 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:10,083 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:10,083 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:10,083 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:10,083 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:10,084 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:10,085 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-15 06:54:10,086 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:10,086 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:10,086 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:10,086 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:10,087 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,087 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:10,087 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,087 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,088 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,088 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:10,088 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,088 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,089 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:10,089 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,089 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:10,089 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,089 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,089 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,089 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-15 06:54:10,090 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:10,090 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:10,091 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444080091 2023-02-15 06:54:10,092 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:10,092 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,094 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:10,095 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:10,095 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:10,095 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444110095 2023-02-15 06:54:10,095 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444170095 2023-02-15 06:54:10,095 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-15 06:54:10,095 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:10,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN}] 2023-02-15 06:54:10,097 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:10,097 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:10,097 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:10,345 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,345 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:10,345 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:10,345 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:10,348 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:10,348 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:10,348 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:10,348 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:10,348 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:10,349 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:10,350 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-15 06:54:10,350 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:10,350 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:10,350 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:10,351 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:10,351 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,351 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:10,351 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,351 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,352 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,352 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:10,352 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,353 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,353 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:10,353 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,353 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:10,354 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,354 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,354 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,354 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-15 06:54:10,354 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:10,355 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:10,355 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444080355 2023-02-15 06:54:10,357 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:10,357 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,359 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:10,359 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:10,359 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:10,359 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444110359 2023-02-15 06:54:10,359 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444170359 2023-02-15 06:54:10,359 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-15 06:54:10,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN}] 2023-02-15 06:54:10,360 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:10,362 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:10,363 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:10,363 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444050363"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444050363"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444050363"}]},"ts":"1676444050363"} 2023-02-15 06:54:10,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:54:10,365 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:10,365 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:10,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:10,365 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:10,610 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,610 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:10,610 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:10,610 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:10,611 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:10,612 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:10,612 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:10,612 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:10,612 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:10,612 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:10,613 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-15 06:54:10,613 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-15 06:54:10,614 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:10,614 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:10,614 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:10,614 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:10,615 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,615 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:10,615 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,615 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,616 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,616 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:10,616 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,616 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,616 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:10,617 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,617 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:10,617 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:10,617 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,617 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,617 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,618 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:10,618 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:10,618 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-15 06:54:10,618 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:10,619 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:10,619 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df to restore RIT 2023-02-15 06:54:10,619 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444080619 2023-02-15 06:54:10,619 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:10,620 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:10,623 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:10,623 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:10,623 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:10,623 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444110623 2023-02-15 06:54:10,623 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444170623 2023-02-15 06:54:10,623 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:10,623 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:10,623 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444050623"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444050623"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444050623"}]},"ts":"1676444050623"} 2023-02-15 06:54:10,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:54:10,625 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:10,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:10,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f6039dce9e5bf374c6575ade89f0e2df, disabling compactions & flushes 2023-02-15 06:54:10,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:10,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:10,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. after waiting 0 ms 2023-02-15 06:54:10,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:10,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing f6039dce9e5bf374c6575ade89f0e2df 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-15 06:54:10,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/7d29f8b696d847d4a0d44c8ac28327cf 2023-02-15 06:54:10,803 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9 2023-02-15 06:54:10,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/7d29f8b696d847d4a0d44c8ac28327cf as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf 2023-02-15 06:54:10,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:54:10,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9 2023-02-15 06:54:10,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:54:10,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f6039dce9e5bf374c6575ade89f0e2df in 39ms, sequenceid=82, compaction requested=true 2023-02-15 06:54:10,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-15 06:54:10,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:10,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:10,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:10,827 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:10,827 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED 2023-02-15 06:54:10,827 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444050827"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444050827"}]},"ts":"1676444050827"} 2023-02-15 06:54:10,829 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:10,829 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:10,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:10,829 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:10,873 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,873 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:10,873 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:10,874 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:10,874 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:10,875 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:10,875 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:10,875 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:10,875 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:10,876 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:10,876 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-15 06:54:10,877 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-15 06:54:10,877 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-15 06:54:10,877 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:10,877 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:10,877 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:10,878 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:10,878 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,878 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:10,879 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,879 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:10,879 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,879 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:10,880 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,880 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:10,880 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:10,881 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,881 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:10,881 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:10,881 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:10,881 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,881 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,881 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:10,881 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:10,881 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:10,881 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:10,882 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-15 06:54:10,882 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:10,883 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:10,883 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df to restore RIT 2023-02-15 06:54:10,888 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444080888 2023-02-15 06:54:10,889 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:10,889 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:10,891 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:10,891 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:10,891 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:10,892 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444110892 2023-02-15 06:54:10,892 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444170892 2023-02-15 06:54:10,892 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-15 06:54:10,892 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:10,892 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444050892"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444050892"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444050892"}]},"ts":"1676444050892"} 2023-02-15 06:54:10,893 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:10,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-15 06:54:10,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 in 268 msec 2023-02-15 06:54:10,896 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:10,896 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:10,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:10,896 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:11,142 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,142 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:11,142 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:11,142 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:11,143 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:11,144 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:11,144 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:11,144 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:11,144 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:11,145 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:11,145 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-15 06:54:11,146 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-15 06:54:11,146 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:11,146 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:11,147 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:11,147 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:11,147 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:11,147 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:11,148 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:11,148 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:11,148 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:11,148 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:11,148 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:11,149 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:11,149 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:11,149 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:11,150 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,150 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:11,150 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:11,150 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:11,150 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,150 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,150 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,151 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:11,151 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:11,151 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-15 06:54:11,151 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:11,152 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:11,152 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df to restore RIT 2023-02-15 06:54:11,156 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444081156 2023-02-15 06:54:11,157 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:11,157 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:11,159 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:11,160 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:11,160 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:11,160 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444111160 2023-02-15 06:54:11,160 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444171160 2023-02-15 06:54:11,160 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:11,160 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:11,162 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:11,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-15 06:54:11,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN in 802 msec 2023-02-15 06:54:11,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:11,169 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:11,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:11,410 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,411 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:11,411 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:11,411 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:11,412 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:11,412 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:11,412 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:11,412 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:11,412 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:11,413 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:11,414 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-15 06:54:11,414 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:11,414 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:11,414 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:11,415 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:11,415 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:11,415 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:11,416 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:11,416 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:11,416 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:11,417 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:11,417 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:11,417 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:11,417 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:11,417 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:11,418 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:11,418 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,419 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:11,419 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:11,419 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:11,419 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:11,419 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:11,419 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-15 06:54:11,420 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:11,421 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:11,426 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444081426 2023-02-15 06:54:11,428 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,428 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:11,431 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:11,431 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:11,432 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:11,432 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444111432 2023-02-15 06:54:11,432 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444171432 2023-02-15 06:54:11,432 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:11,432 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,432 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,432 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,438 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:11,449 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=f6039dce9e5bf374c6575ade89f0e2df, threads=14 2023-02-15 06:54:11,450 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,450 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,450 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,450 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,450 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,451 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,451 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,456 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,456 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,456 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,457 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,457 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,457 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,458 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,696 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,718 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,728 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,728 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,740 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,756 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,767 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,767 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,772 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,772 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,773 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,779 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,793 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,794 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,794 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region f6039dce9e5bf374c6575ade89f0e2df Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-15 06:54:11,835 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:11,835 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:11,835 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:11,932 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,932 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:11,932 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:11,932 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:11,933 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:11,933 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:11,933 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:11,933 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:11,933 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:11,934 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:11,935 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-15 06:54:11,935 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:11,935 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:11,935 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:11,935 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:11,936 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:11,936 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:11,936 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:11,936 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:11,936 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:11,936 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:11,937 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:11,937 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:11,937 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:11,937 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:11,937 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:11,938 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,938 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:11,938 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:11,938 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:11,938 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,938 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,938 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,938 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:11,939 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:11,939 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-15 06:54:11,939 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:11,940 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:11,940 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444081940 2023-02-15 06:54:11,942 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:11,942 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:11,944 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:11,944 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:11,944 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:11,944 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444111944 2023-02-15 06:54:11,945 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444171945 2023-02-15 06:54:11,945 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:11,953 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=f6039dce9e5bf374c6575ade89f0e2df, threads=14 2023-02-15 06:54:11,954 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,954 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,954 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,955 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,955 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,955 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,955 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,956 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,956 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,957 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,958 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,959 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,960 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:11,963 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,115 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,135 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-15 06:54:12,136 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,154 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,155 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,168 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,216 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,224 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,224 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,224 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,229 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,242 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,248 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,259 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,260 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098 for region: f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:12,260 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region f6039dce9e5bf374c6575ade89f0e2df Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-15 06:54:12,307 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:12,313 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/d7c164e31035871a9d949fd6bf789f4d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:54:12,315 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/876df0ac8f4998d11d37e33e51b4f60a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:54:12,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:12,315 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:12,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:12,445 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,445 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:12,445 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:12,445 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:12,447 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:12,447 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:12,447 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:12,447 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:12,447 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:12,448 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:12,449 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-15 06:54:12,449 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:12,449 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:12,449 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:12,450 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:12,450 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:12,450 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:12,450 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:12,450 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:12,451 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:12,451 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:12,451 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:12,451 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:12,451 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:12,451 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:12,452 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:12,452 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,452 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:12,452 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:12,456 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:12,456 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,456 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,456 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,456 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:12,456 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:12,456 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-15 06:54:12,457 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:12,458 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:12,460 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444082460 2023-02-15 06:54:12,460 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:12,462 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,464 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:12,464 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:12,464 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:12,464 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444112464 2023-02-15 06:54:12,465 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444172465 2023-02-15 06:54:12,465 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-15 06:54:12,467 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:12,471 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:12,471 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:12,471 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:12,716 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,716 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:12,716 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:12,716 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:12,717 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:12,717 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:12,718 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:12,718 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:12,718 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:12,719 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:12,720 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-15 06:54:12,720 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:12,720 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:12,720 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:12,721 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:12,721 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:12,721 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:12,722 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:12,722 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:12,722 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:12,723 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:12,723 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:12,723 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:12,724 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:12,724 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:12,724 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:12,725 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,725 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:12,725 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:12,725 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:12,725 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,726 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,726 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,726 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:12,726 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:12,726 DEBUG [Listener at localhost/37669] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,726 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-15 06:54:12,727 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:12,727 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:12,728 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444082728 2023-02-15 06:54:12,730 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,730 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:12,733 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:12,733 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:12,733 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:12,734 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444112734 2023-02-15 06:54:12,734 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444172734 2023-02-15 06:54:12,734 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:12,734 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:12,746 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:12,746 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-15 06:54:12,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:12,746 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:12,985 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,985 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:12,985 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:12,985 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:12,985 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:12,986 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:12,986 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:12,986 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:12,986 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:12,987 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:12,988 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-15 06:54:12,988 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:12,988 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:12,988 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:12,988 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:12,989 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:12,989 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:12,989 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:12,989 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:12,989 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:12,990 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:12,990 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:12,990 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:12,990 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:12,990 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:12,991 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:12,992 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,992 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:12,992 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:12,992 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:12,992 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,992 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,992 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,992 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:12,992 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:12,993 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-15 06:54:12,993 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:12,994 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:12,994 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444082994 2023-02-15 06:54:12,997 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:12,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:12,999 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:12,999 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:12,999 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:12,999 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444112999 2023-02-15 06:54:12,999 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444172999 2023-02-15 06:54:12,999 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-15 06:54:13,000 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:13,002 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:13,002 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-15 06:54:13,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:13,002 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:13,250 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,251 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:13,251 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:13,251 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:13,251 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:13,251 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:13,251 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:13,251 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:13,251 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:13,252 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:13,253 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-15 06:54:13,253 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:13,253 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:13,253 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:13,253 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:13,254 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:13,254 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:13,254 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:13,254 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:13,254 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:13,255 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:13,255 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:13,255 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:13,255 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:13,255 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:13,255 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:13,256 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,256 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:13,256 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:13,257 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:13,257 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,257 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,257 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,257 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:13,257 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:13,257 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-15 06:54:13,257 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:13,258 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:13,258 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444083258 2023-02-15 06:54:13,260 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:13,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,262 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:13,263 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:13,263 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:13,263 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444113263 2023-02-15 06:54:13,263 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444173263 2023-02-15 06:54:13,263 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:13,263 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:13,266 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:13,266 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-15 06:54:13,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:13,266 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:13,513 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,514 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:13,514 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:13,514 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:13,515 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:13,515 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:13,515 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:13,515 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:13,515 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:13,516 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:13,517 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-15 06:54:13,517 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:13,517 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:13,517 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:13,517 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:13,517 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:13,517 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:13,518 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:13,518 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:13,518 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:13,518 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:13,519 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:13,519 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:13,519 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:13,519 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:13,519 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:13,520 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,520 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:13,520 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:13,520 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:13,520 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,520 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,520 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,520 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:13,520 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:13,520 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-15 06:54:13,521 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:13,521 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:13,522 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444083522 2023-02-15 06:54:13,523 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:13,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,525 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:13,525 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:13,525 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:13,525 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444113525 2023-02-15 06:54:13,526 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444173526 2023-02-15 06:54:13,526 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:13,526 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:13,529 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN] 2023-02-15 06:54:13,529 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:13,529 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-15 06:54:13,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:13,529 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:13,776 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,776 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:13,776 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:13,776 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:13,777 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:13,777 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:13,777 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:13,777 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:13,777 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:13,778 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:13,779 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-15 06:54:13,779 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:54:13,779 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:54:13,779 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-15 06:54:13,779 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:13,779 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:13,780 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:13,780 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:13,780 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:13,780 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:13,780 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:13,780 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:13,781 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:13,781 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:13,781 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:13,781 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:13,781 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:13,782 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,782 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:13,782 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:13,782 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:13,783 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:13,783 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,783 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,783 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,783 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:13,783 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:13,783 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:13,783 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-15 06:54:13,784 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:13,784 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:13,784 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df to restore RIT 2023-02-15 06:54:13,784 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444083784 2023-02-15 06:54:13,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:13,787 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:13,788 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:13,790 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:13,790 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:13,790 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:13,790 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444113790 2023-02-15 06:54:13,790 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444173790 2023-02-15 06:54:13,790 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:13,791 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:13,796 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:13,796 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-15 06:54:13,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:13,796 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:14,041 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,041 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:14,041 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:14,041 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:14,043 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:14,043 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:14,044 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:14,044 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:14,044 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:14,044 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:14,045 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-15 06:54:14,045 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-15 06:54:14,045 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:14,046 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:14,046 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:14,046 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:14,046 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,046 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:14,047 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,047 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,047 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,047 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:14,047 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,047 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,048 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:14,048 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,048 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,048 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:14,048 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,048 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,048 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,048 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:14,048 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-15 06:54:14,049 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:14,049 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:14,049 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df to restore RIT 2023-02-15 06:54:14,050 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444084050 2023-02-15 06:54:14,051 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:14,051 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,052 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,054 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:14,054 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:14,054 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:14,054 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444114054 2023-02-15 06:54:14,054 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444174054 2023-02-15 06:54:14,055 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:14,055 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:14,057 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:14,057 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-15 06:54:14,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:14,057 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:14,305 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,305 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:14,305 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:14,305 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:14,305 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:14,305 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:14,305 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:14,305 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:14,305 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:14,306 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:14,307 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-15 06:54:14,307 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-15 06:54:14,307 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:14,308 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:14,308 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:14,308 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:14,309 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,309 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:14,309 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,309 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,310 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,310 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:14,310 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,310 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,311 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:14,311 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,311 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,311 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:14,311 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,312 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,312 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,312 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:14,312 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-15 06:54:14,313 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:14,313 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:14,313 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df to restore RIT 2023-02-15 06:54:14,314 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444084314 2023-02-15 06:54:14,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,316 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:14,317 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,318 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:14,319 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:14,319 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:14,319 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444114319 2023-02-15 06:54:14,319 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444174319 2023-02-15 06:54:14,319 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:54:14,319 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:14,321 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:14,321 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-15 06:54:14,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:14,321 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:14,569 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,569 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:14,569 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:14,569 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:14,569 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:14,570 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:14,570 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:14,570 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:14,570 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:14,570 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:14,571 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-15 06:54:14,571 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-15 06:54:14,571 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:14,572 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:14,572 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:14,572 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:14,573 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,573 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:14,573 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,573 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,574 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,574 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:14,574 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,574 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,575 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:14,575 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,575 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,575 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:14,575 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,576 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,576 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,576 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:54:14,576 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-15 06:54:14,577 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:14,577 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:14,577 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df to restore RIT 2023-02-15 06:54:14,578 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444084578 2023-02-15 06:54:14,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,580 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:14,582 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,584 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:14,585 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:14,585 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:14,585 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444114585 2023-02-15 06:54:14,585 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444174585 2023-02-15 06:54:14,585 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-15 06:54:14,585 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:54:14,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a exec-time=5.0210 sec 2023-02-15 06:54:14,588 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,589 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,35089,1676444032716; forceNewPlan=false, retain=true 2023-02-15 06:54:14,739 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:54:14,740 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:14,740 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444054740"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444054740"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444054740"}]},"ts":"1676444054740"} 2023-02-15 06:54:14,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:54:14,742 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:54:14,742 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:54:14,742 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:54:14,835 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:14,835 WARN [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-15 06:54:14,835 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:54:14,835 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:54:14,835 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:54:14,836 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:54:14,836 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:54:14,836 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:54:14,836 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:54:14,836 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:54:14,837 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:54:14,838 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-15 06:54:14,838 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:54:14,838 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:54:14,838 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:54:14,838 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:54:14,839 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,839 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:54:14,839 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,839 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:54:14,839 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,840 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:54:14,840 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,840 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:54:14,840 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:14,840 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:54:14,840 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,840 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:54:14,841 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,841 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-15 06:54:14,841 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:54:14,842 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:54:14,842 INFO [Listener at localhost/37669] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df to restore RIT 2023-02-15 06:54:14,842 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444084842 2023-02-15 06:54:14,843 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:54:14,843 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:54:14,845 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:14,845 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f6039dce9e5bf374c6575ade89f0e2df, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:54:14,846 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:54:14,846 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444114846 2023-02-15 06:54:14,846 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444174846 2023-02-15 06:54:14,846 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-15 06:54:14,846 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:14,846 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444054846"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444054846"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444054846"}]},"ts":"1676444054846"} 2023-02-15 06:54:14,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:54:15,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:15,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f6039dce9e5bf374c6575ade89f0e2df, NAME => 'testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:54:15,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:15,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:15,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,005 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,005 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1 2023-02-15 06:54:15,006 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1 2023-02-15 06:54:15,006 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f6039dce9e5bf374c6575ade89f0e2df columnFamilyName cf1 2023-02-15 06:54:15,012 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482 2023-02-15 06:54:15,015 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb 2023-02-15 06:54:15,018 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143 2023-02-15 06:54:15,020 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa 2023-02-15 06:54:15,023 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf 2023-02-15 06:54:15,026 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d 2023-02-15 06:54:15,030 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098 2023-02-15 06:54:15,030 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(310): Store=f6039dce9e5bf374c6575ade89f0e2df/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:15,030 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,031 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2 2023-02-15 06:54:15,031 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2 2023-02-15 06:54:15,031 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f6039dce9e5bf374c6575ade89f0e2df columnFamilyName cf2 2023-02-15 06:54:15,037 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b 2023-02-15 06:54:15,041 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00 2023-02-15 06:54:15,044 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222 2023-02-15 06:54:15,048 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827 2023-02-15 06:54:15,051 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8 2023-02-15 06:54:15,055 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea 2023-02-15 06:54:15,058 DEBUG [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9 2023-02-15 06:54:15,058 INFO [StoreOpener-f6039dce9e5bf374c6575ade89f0e2df-1] regionserver.HStore(310): Store=f6039dce9e5bf374c6575ade89f0e2df/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:15,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.splits 2023-02-15 06:54:15,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:15,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f6039dce9e5bf374c6575ade89f0e2df; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67650216, jitterRate=0.008066773414611816}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:15,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:15,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df., pid=48, masterSystemTime=1676444055000 2023-02-15 06:54:15,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:15,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:54:15,066 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:54:15,066 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:54:15,067 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-15 06:54:15,067 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-15 06:54:15,067 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HStore(1912): f6039dce9e5bf374c6575ade89f0e2df/cf1 is initiating minor compaction (all files) 2023-02-15 06:54:15,067 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HStore(1912): f6039dce9e5bf374c6575ade89f0e2df/cf2 is initiating minor compaction (all files) 2023-02-15 06:54:15,067 INFO [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f6039dce9e5bf374c6575ade89f0e2df/cf1 in testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:15,067 INFO [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f6039dce9e5bf374c6575ade89f0e2df/cf2 in testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:15,068 INFO [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp, totalSize=34.9 K 2023-02-15 06:54:15,068 INFO [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp, totalSize=34.9 K 2023-02-15 06:54:15,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:15,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:15,069 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting 4ddff29601ed48c89f1d180c026b0143, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444048133 2023-02-15 06:54:15,069 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting 56a140bece9d457a873231c410fc1aa8, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444048133 2023-02-15 06:54:15,069 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:15,069 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting 3c3102a38a0f48f481e35ee24e114482, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444048322 2023-02-15 06:54:15,069 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting 569a18e0ece64733b9d60678c869e827, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444048322 2023-02-15 06:54:15,069 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444055069"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444055069"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444055069"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444055069"}]},"ts":"1676444055069"} 2023-02-15 06:54:15,070 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting 9cd8800f408f46188974b73872a75d1d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444048529 2023-02-15 06:54:15,070 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting 71da874e4fb8428185831cde722d7cea, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444048529 2023-02-15 06:54:15,070 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting af286c32355a410998b8d4944b538098, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444048725 2023-02-15 06:54:15,070 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting 1d49ba87102b4b8aae24c629324c945b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444048725 2023-02-15 06:54:15,071 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting 530190ce10fb48db8924ad24178c76fa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444048918 2023-02-15 06:54:15,071 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting 335613eacaf84668bd7e8a6d1f189222, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444048918 2023-02-15 06:54:15,071 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting 410fa16390d6437ab1f2a18cb643aebb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444049106 2023-02-15 06:54:15,071 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting 1ffbf9cf284c446f981fb23311466f00, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444049106 2023-02-15 06:54:15,072 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting 7d29f8b696d847d4a0d44c8ac28327cf, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444049301 2023-02-15 06:54:15,072 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting bff5a3a1d6b44bfeb0e00b75531e37e9, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444049301 2023-02-15 06:54:15,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-15 06:54:15,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 in 224 msec 2023-02-15 06:54:15,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN in 1.5460 sec 2023-02-15 06:54:15,095 INFO [RS:0;jenkins-hbase4:35089-shortCompactions-0] throttle.PressureAwareThroughputController(145): f6039dce9e5bf374c6575ade89f0e2df#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:15,099 INFO [RS:0;jenkins-hbase4:35089-longCompactions-0] throttle.PressureAwareThroughputController(145): f6039dce9e5bf374c6575ade89f0e2df#cf2#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:15,116 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf1/3be220fbfe1d45b4ba275975ee54e2ae as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3be220fbfe1d45b4ba275975ee54e2ae 2023-02-15 06:54:15,117 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/.tmp/cf2/6c49c4c9f1094272ab6d33fe282942ee as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/6c49c4c9f1094272ab6d33fe282942ee 2023-02-15 06:54:15,123 INFO [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f6039dce9e5bf374c6575ade89f0e2df/cf2 of f6039dce9e5bf374c6575ade89f0e2df into 6c49c4c9f1094272ab6d33fe282942ee(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:15,123 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:15,123 INFO [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df., storeName=f6039dce9e5bf374c6575ade89f0e2df/cf2, priority=9, startTime=1676444055065; duration=0sec 2023-02-15 06:54:15,123 INFO [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f6039dce9e5bf374c6575ade89f0e2df/cf1 of f6039dce9e5bf374c6575ade89f0e2df into 3be220fbfe1d45b4ba275975ee54e2ae(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:15,123 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:15,123 INFO [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df., storeName=f6039dce9e5bf374c6575ade89f0e2df/cf1, priority=9, startTime=1676444055065; duration=0sec 2023-02-15 06:54:15,125 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:15,125 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:15,423 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-15 06:54:15,425 DEBUG [Listener at localhost/37669] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-15 06:54:15,427 INFO [Listener at localhost/37669] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-15 06:54:15,429 INFO [Listener at localhost/37669] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-15 06:54:15,429 INFO [Listener at localhost/37669] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-15 06:54:15,429 INFO [Listener at localhost/37669] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-15 06:54:15,442 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:15,444 INFO [Listener at localhost/37669] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-15 06:54:15,444 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-15 06:54:15,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:15,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-15 06:54:15,448 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444055448"}]},"ts":"1676444055448"} 2023-02-15 06:54:15,449 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-15 06:54:15,450 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-15 06:54:15,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN}] 2023-02-15 06:54:15,452 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN 2023-02-15 06:54:15,452 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:15,453 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444055452"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444055452"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444055452"}]},"ts":"1676444055452"} 2023-02-15 06:54:15,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:54:15,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-15 06:54:15,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f6039dce9e5bf374c6575ade89f0e2df, disabling compactions & flushes 2023-02-15 06:54:15,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:15,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:15,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. after waiting 0 ms 2023-02-15 06:54:15,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:15,607 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf] to archive 2023-02-15 06:54:15,608 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:15,609 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/4ddff29601ed48c89f1d180c026b0143 2023-02-15 06:54:15,610 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3c3102a38a0f48f481e35ee24e114482 2023-02-15 06:54:15,611 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/9cd8800f408f46188974b73872a75d1d 2023-02-15 06:54:15,612 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/af286c32355a410998b8d4944b538098 2023-02-15 06:54:15,614 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/530190ce10fb48db8924ad24178c76fa 2023-02-15 06:54:15,615 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/410fa16390d6437ab1f2a18cb643aebb 2023-02-15 06:54:15,616 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/7d29f8b696d847d4a0d44c8ac28327cf 2023-02-15 06:54:15,620 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9] to archive 2023-02-15 06:54:15,621 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:15,622 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/56a140bece9d457a873231c410fc1aa8 2023-02-15 06:54:15,623 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/569a18e0ece64733b9d60678c869e827 2023-02-15 06:54:15,625 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/71da874e4fb8428185831cde722d7cea 2023-02-15 06:54:15,626 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1d49ba87102b4b8aae24c629324c945b 2023-02-15 06:54:15,627 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/335613eacaf84668bd7e8a6d1f189222 2023-02-15 06:54:15,628 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/1ffbf9cf284c446f981fb23311466f00 2023-02-15 06:54:15,629 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/bff5a3a1d6b44bfeb0e00b75531e37e9 2023-02-15 06:54:15,632 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-15 06:54:15,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:15,633 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df. 2023-02-15 06:54:15,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f6039dce9e5bf374c6575ade89f0e2df: 2023-02-15 06:54:15,634 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,635 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=f6039dce9e5bf374c6575ade89f0e2df, regionState=CLOSED 2023-02-15 06:54:15,635 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444055635"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444055635"}]},"ts":"1676444055635"} 2023-02-15 06:54:15,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-15 06:54:15,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure f6039dce9e5bf374c6575ade89f0e2df, server=jenkins-hbase4.apache.org,35089,1676444032716 in 182 msec 2023-02-15 06:54:15,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-15 06:54:15,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, UNASSIGN in 187 msec 2023-02-15 06:54:15,640 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444055640"}]},"ts":"1676444055640"} 2023-02-15 06:54:15,642 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-15 06:54:15,643 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-15 06:54:15,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 200 msec 2023-02-15 06:54:15,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-15 06:54:15,751 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-15 06:54:15,752 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-15 06:54:15,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:15,754 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:15,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-15 06:54:15,756 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:15,759 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,761 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/recovered.edits] 2023-02-15 06:54:15,764 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3be220fbfe1d45b4ba275975ee54e2ae to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf1/3be220fbfe1d45b4ba275975ee54e2ae 2023-02-15 06:54:15,766 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/6c49c4c9f1094272ab6d33fe282942ee to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/cf2/6c49c4c9f1094272ab6d33fe282942ee 2023-02-15 06:54:15,769 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/recovered.edits/90.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df/recovered.edits/90.seqid 2023-02-15 06:54:15,769 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackAndDoubleExecution/f6039dce9e5bf374c6575ade89f0e2df 2023-02-15 06:54:15,769 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-15 06:54:15,772 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:15,774 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-15 06:54:15,776 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-15 06:54:15,777 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:15,777 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-15 06:54:15,777 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444055777"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:15,779 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-15 06:54:15,779 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f6039dce9e5bf374c6575ade89f0e2df, NAME => 'testRollbackAndDoubleExecution,,1676444047625.f6039dce9e5bf374c6575ade89f0e2df.', STARTKEY => '', ENDKEY => ''}] 2023-02-15 06:54:15,779 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-15 06:54:15,779 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676444055779"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:15,780 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-15 06:54:15,782 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:54:15,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 30 msec 2023-02-15 06:54:15,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-15 06:54:15,857 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-15 06:54:15,872 INFO [Listener at localhost/37669] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=397 (was 395) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:36676 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:36680 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:36124 [Waiting for operation #49] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:35089-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:53490 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:40118 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:53498 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:53456 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:36666 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:40084 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:36652 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:49230 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:53594 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:36010 [Waiting for operation #49] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:49238 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:53672 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_366043887_15 at /127.0.0.1:40090 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=836 (was 859), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=262 (was 276), ProcessCount=168 (was 168), AvailableMemoryMB=3308 (was 3277) - AvailableMemoryMB LEAK? - 2023-02-15 06:54:15,886 INFO [Listener at localhost/37669] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=397, OpenFileDescriptor=836, MaxFileDescriptor=60000, SystemLoadAverage=262, ProcessCount=168, AvailableMemoryMB=3307 2023-02-15 06:54:15,886 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:15,890 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-15 06:54:15,890 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-15 06:54:15,891 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:54:15,893 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:54:15,894 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:54:15,895 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:15,896 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032 empty. 2023-02-15 06:54:15,896 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:15,897 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-15 06:54:15,907 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-15 06:54:15,908 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8938ff632984211f5fe6b773e2e60032, NAME => 'testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:54:15,916 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:15,916 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 8938ff632984211f5fe6b773e2e60032, disabling compactions & flushes 2023-02-15 06:54:15,916 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:15,916 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:15,916 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. after waiting 0 ms 2023-02-15 06:54:15,916 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:15,916 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:15,916 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 8938ff632984211f5fe6b773e2e60032: 2023-02-15 06:54:15,918 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:54:15,919 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676444055919"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444055919"}]},"ts":"1676444055919"} 2023-02-15 06:54:15,921 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:54:15,922 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:54:15,922 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444055922"}]},"ts":"1676444055922"} 2023-02-15 06:54:15,923 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-15 06:54:15,926 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:54:15,927 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:54:15,927 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:54:15,927 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:54:15,927 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:54:15,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8938ff632984211f5fe6b773e2e60032, ASSIGN}] 2023-02-15 06:54:15,929 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8938ff632984211f5fe6b773e2e60032, ASSIGN 2023-02-15 06:54:15,930 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8938ff632984211f5fe6b773e2e60032, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40475,1676444032771; forceNewPlan=false, retain=false 2023-02-15 06:54:16,080 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:54:16,081 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=8938ff632984211f5fe6b773e2e60032, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:16,082 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676444056081"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444056081"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444056081"}]},"ts":"1676444056081"} 2023-02-15 06:54:16,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 8938ff632984211f5fe6b773e2e60032, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:16,236 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:16,236 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-15 06:54:16,239 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59798, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-15 06:54:16,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:16,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8938ff632984211f5fe6b773e2e60032, NAME => 'testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:54:16,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:16,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:16,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:16,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:16,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:16,247 INFO [StoreOpener-8938ff632984211f5fe6b773e2e60032-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:16,248 DEBUG [StoreOpener-8938ff632984211f5fe6b773e2e60032-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/cf1 2023-02-15 06:54:16,248 DEBUG [StoreOpener-8938ff632984211f5fe6b773e2e60032-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/cf1 2023-02-15 06:54:16,248 INFO [StoreOpener-8938ff632984211f5fe6b773e2e60032-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8938ff632984211f5fe6b773e2e60032 columnFamilyName cf1 2023-02-15 06:54:16,249 INFO [StoreOpener-8938ff632984211f5fe6b773e2e60032-1] regionserver.HStore(310): Store=8938ff632984211f5fe6b773e2e60032/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:16,249 INFO [StoreOpener-8938ff632984211f5fe6b773e2e60032-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:16,251 DEBUG [StoreOpener-8938ff632984211f5fe6b773e2e60032-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/cf2 2023-02-15 06:54:16,251 DEBUG [StoreOpener-8938ff632984211f5fe6b773e2e60032-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/cf2 2023-02-15 06:54:16,251 INFO [StoreOpener-8938ff632984211f5fe6b773e2e60032-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8938ff632984211f5fe6b773e2e60032 columnFamilyName cf2 2023-02-15 06:54:16,252 INFO [StoreOpener-8938ff632984211f5fe6b773e2e60032-1] regionserver.HStore(310): Store=8938ff632984211f5fe6b773e2e60032/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:16,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:16,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:16,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:16,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:16,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:54:16,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8938ff632984211f5fe6b773e2e60032; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65092846, jitterRate=-0.030041009187698364}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:16,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8938ff632984211f5fe6b773e2e60032: 2023-02-15 06:54:16,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032., pid=55, masterSystemTime=1676444056236 2023-02-15 06:54:16,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:16,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:16,263 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=8938ff632984211f5fe6b773e2e60032, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:16,263 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676444056263"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444056263"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444056263"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444056263"}]},"ts":"1676444056263"} 2023-02-15 06:54:16,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-15 06:54:16,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 8938ff632984211f5fe6b773e2e60032, server=jenkins-hbase4.apache.org,40475,1676444032771 in 182 msec 2023-02-15 06:54:16,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-15 06:54:16,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8938ff632984211f5fe6b773e2e60032, ASSIGN in 339 msec 2023-02-15 06:54:16,269 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:54:16,269 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444056269"}]},"ts":"1676444056269"} 2023-02-15 06:54:16,270 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-15 06:54:16,272 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:54:16,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 381 msec 2023-02-15 06:54:16,393 DEBUG [Listener at localhost/37669] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:16,394 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,395 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,395 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,395 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8938ff632984211f5fe6b773e2e60032, UNASSIGN}] 2023-02-15 06:54:16,398 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8938ff632984211f5fe6b773e2e60032, UNASSIGN 2023-02-15 06:54:16,399 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=8938ff632984211f5fe6b773e2e60032, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:16,399 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676444056399"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444056399"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444056399"}]},"ts":"1676444056399"} 2023-02-15 06:54:16,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 8938ff632984211f5fe6b773e2e60032, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:16,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:16,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8938ff632984211f5fe6b773e2e60032, disabling compactions & flushes 2023-02-15 06:54:16,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:16,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:16,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. after waiting 0 ms 2023-02-15 06:54:16,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:16,557 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-15 06:54:16,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:16,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032. 2023-02-15 06:54:16,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8938ff632984211f5fe6b773e2e60032: 2023-02-15 06:54:16,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:16,560 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=8938ff632984211f5fe6b773e2e60032, regionState=CLOSED 2023-02-15 06:54:16,560 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676444056560"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444056560"}]},"ts":"1676444056560"} 2023-02-15 06:54:16,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-15 06:54:16,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 8938ff632984211f5fe6b773e2e60032, server=jenkins-hbase4.apache.org,40475,1676444032771 in 161 msec 2023-02-15 06:54:16,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-15 06:54:16,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8938ff632984211f5fe6b773e2e60032, UNASSIGN in 166 msec 2023-02-15 06:54:16,588 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-15 06:54:16,590 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-15 06:54:16,591 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676444056591"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676444056591"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676444056591"}]},"ts":"1676444056591"} 2023-02-15 06:54:16,591 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444056591"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444056591"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444056591"}]},"ts":"1676444056591"} 2023-02-15 06:54:16,591 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444056591"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444056591"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444056591"}]},"ts":"1676444056591"} 2023-02-15 06:54:16,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=38f136c511f939e3641e736608f6d72a, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=83267c2682581a47fe295685701d02ca, ASSIGN}] 2023-02-15 06:54:16,596 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=83267c2682581a47fe295685701d02ca, ASSIGN 2023-02-15 06:54:16,596 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=83267c2682581a47fe295685701d02ca, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40475,1676444032771; forceNewPlan=false, retain=false 2023-02-15 06:54:16,597 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=38f136c511f939e3641e736608f6d72a, ASSIGN 2023-02-15 06:54:16,597 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=38f136c511f939e3641e736608f6d72a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40475,1676444032771; forceNewPlan=false, retain=false 2023-02-15 06:54:16,746 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-15 06:54:16,748 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=83267c2682581a47fe295685701d02ca, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:16,748 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444056748"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444056748"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444056748"}]},"ts":"1676444056748"} 2023-02-15 06:54:16,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure 83267c2682581a47fe295685701d02ca, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:16,752 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=38f136c511f939e3641e736608f6d72a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:16,753 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444056752"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444056752"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444056752"}]},"ts":"1676444056752"} 2023-02-15 06:54:16,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure 38f136c511f939e3641e736608f6d72a, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:16,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca. 2023-02-15 06:54:16,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 83267c2682581a47fe295685701d02ca, NAME => 'testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca.', STARTKEY => '41', ENDKEY => ''} 2023-02-15 06:54:16,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:16,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:16,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,907 INFO [StoreOpener-83267c2682581a47fe295685701d02ca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,908 DEBUG [StoreOpener-83267c2682581a47fe295685701d02ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/cf1 2023-02-15 06:54:16,908 DEBUG [StoreOpener-83267c2682581a47fe295685701d02ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/cf1 2023-02-15 06:54:16,909 INFO [StoreOpener-83267c2682581a47fe295685701d02ca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 83267c2682581a47fe295685701d02ca columnFamilyName cf1 2023-02-15 06:54:16,909 INFO [StoreOpener-83267c2682581a47fe295685701d02ca-1] regionserver.HStore(310): Store=83267c2682581a47fe295685701d02ca/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:16,909 INFO [StoreOpener-83267c2682581a47fe295685701d02ca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,910 DEBUG [StoreOpener-83267c2682581a47fe295685701d02ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/cf2 2023-02-15 06:54:16,910 DEBUG [StoreOpener-83267c2682581a47fe295685701d02ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/cf2 2023-02-15 06:54:16,911 INFO [StoreOpener-83267c2682581a47fe295685701d02ca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 83267c2682581a47fe295685701d02ca columnFamilyName cf2 2023-02-15 06:54:16,911 INFO [StoreOpener-83267c2682581a47fe295685701d02ca-1] regionserver.HStore(310): Store=83267c2682581a47fe295685701d02ca/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:16,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:16,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 83267c2682581a47fe295685701d02ca 2023-02-15 06:54:16,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 83267c2682581a47fe295685701d02ca; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71703481, jitterRate=0.06846512854099274}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:16,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 83267c2682581a47fe295685701d02ca: 2023-02-15 06:54:16,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca., pid=61, masterSystemTime=1676444056902 2023-02-15 06:54:16,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca. 2023-02-15 06:54:16,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca. 2023-02-15 06:54:16,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a. 2023-02-15 06:54:16,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 38f136c511f939e3641e736608f6d72a, NAME => 'testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a.', STARTKEY => '', ENDKEY => '41'} 2023-02-15 06:54:16,919 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=83267c2682581a47fe295685701d02ca, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:16,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:16,919 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444056919"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444056919"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444056919"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444056919"}]},"ts":"1676444056919"} 2023-02-15 06:54:16,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:16,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:16,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:16,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:16,921 INFO [StoreOpener-38f136c511f939e3641e736608f6d72a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:16,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-15 06:54:16,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure 83267c2682581a47fe295685701d02ca, server=jenkins-hbase4.apache.org,40475,1676444032771 in 171 msec 2023-02-15 06:54:16,922 DEBUG [StoreOpener-38f136c511f939e3641e736608f6d72a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/cf1 2023-02-15 06:54:16,922 DEBUG [StoreOpener-38f136c511f939e3641e736608f6d72a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/cf1 2023-02-15 06:54:16,923 INFO [StoreOpener-38f136c511f939e3641e736608f6d72a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 38f136c511f939e3641e736608f6d72a columnFamilyName cf1 2023-02-15 06:54:16,923 INFO [StoreOpener-38f136c511f939e3641e736608f6d72a-1] regionserver.HStore(310): Store=38f136c511f939e3641e736608f6d72a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:16,924 INFO [StoreOpener-38f136c511f939e3641e736608f6d72a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:16,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=83267c2682581a47fe295685701d02ca, ASSIGN in 327 msec 2023-02-15 06:54:16,925 DEBUG [StoreOpener-38f136c511f939e3641e736608f6d72a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/cf2 2023-02-15 06:54:16,925 DEBUG [StoreOpener-38f136c511f939e3641e736608f6d72a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/cf2 2023-02-15 06:54:16,925 INFO [StoreOpener-38f136c511f939e3641e736608f6d72a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 38f136c511f939e3641e736608f6d72a columnFamilyName cf2 2023-02-15 06:54:16,926 INFO [StoreOpener-38f136c511f939e3641e736608f6d72a-1] regionserver.HStore(310): Store=38f136c511f939e3641e736608f6d72a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:16,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:16,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:16,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:16,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:16,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 38f136c511f939e3641e736608f6d72a; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74447397, jitterRate=0.10935266315937042}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:16,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 38f136c511f939e3641e736608f6d72a: 2023-02-15 06:54:16,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a., pid=62, masterSystemTime=1676444056902 2023-02-15 06:54:16,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a. 2023-02-15 06:54:16,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a. 2023-02-15 06:54:16,933 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=38f136c511f939e3641e736608f6d72a, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:16,933 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444056933"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444056933"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444056933"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444056933"}]},"ts":"1676444056933"} 2023-02-15 06:54:16,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=59 2023-02-15 06:54:16,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure 38f136c511f939e3641e736608f6d72a, server=jenkins-hbase4.apache.org,40475,1676444032771 in 181 msec 2023-02-15 06:54:16,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-15 06:54:16,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=38f136c511f939e3641e736608f6d72a, ASSIGN in 342 msec 2023-02-15 06:54:16,939 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca in 545 msec 2023-02-15 06:54:17,147 DEBUG [hconnection-0x7e2f5f25-shared-pool-11] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-15 06:54:17,151 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59810, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-15 06:54:17,153 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-15 06:54:17,156 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:17,158 INFO [Listener at localhost/37669] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-15 06:54:17,158 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-15 06:54:17,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:54:17,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-15 06:54:17,162 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444057162"}]},"ts":"1676444057162"} 2023-02-15 06:54:17,163 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-15 06:54:17,165 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-15 06:54:17,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=38f136c511f939e3641e736608f6d72a, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=83267c2682581a47fe295685701d02ca, UNASSIGN}] 2023-02-15 06:54:17,167 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=83267c2682581a47fe295685701d02ca, UNASSIGN 2023-02-15 06:54:17,168 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=83267c2682581a47fe295685701d02ca, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:17,168 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444057168"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444057168"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444057168"}]},"ts":"1676444057168"} 2023-02-15 06:54:17,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 83267c2682581a47fe295685701d02ca, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:17,172 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=38f136c511f939e3641e736608f6d72a, UNASSIGN 2023-02-15 06:54:17,172 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=38f136c511f939e3641e736608f6d72a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:17,173 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444057172"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444057172"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444057172"}]},"ts":"1676444057172"} 2023-02-15 06:54:17,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 38f136c511f939e3641e736608f6d72a, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:17,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-15 06:54:17,321 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 83267c2682581a47fe295685701d02ca 2023-02-15 06:54:17,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 83267c2682581a47fe295685701d02ca, disabling compactions & flushes 2023-02-15 06:54:17,322 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca. 2023-02-15 06:54:17,322 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca. 2023-02-15 06:54:17,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca. after waiting 0 ms 2023-02-15 06:54:17,323 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca. 2023-02-15 06:54:17,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-15 06:54:17,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:17,327 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca. 2023-02-15 06:54:17,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 83267c2682581a47fe295685701d02ca: 2023-02-15 06:54:17,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 83267c2682581a47fe295685701d02ca 2023-02-15 06:54:17,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:17,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 38f136c511f939e3641e736608f6d72a, disabling compactions & flushes 2023-02-15 06:54:17,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a. 2023-02-15 06:54:17,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a. 2023-02-15 06:54:17,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a. after waiting 0 ms 2023-02-15 06:54:17,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a. 2023-02-15 06:54:17,330 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=83267c2682581a47fe295685701d02ca, regionState=CLOSED 2023-02-15 06:54:17,331 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444057330"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444057330"}]},"ts":"1676444057330"} 2023-02-15 06:54:17,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-15 06:54:17,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-15 06:54:17,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 83267c2682581a47fe295685701d02ca, server=jenkins-hbase4.apache.org,40475,1676444032771 in 163 msec 2023-02-15 06:54:17,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:17,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a. 2023-02-15 06:54:17,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 38f136c511f939e3641e736608f6d72a: 2023-02-15 06:54:17,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=83267c2682581a47fe295685701d02ca, UNASSIGN in 169 msec 2023-02-15 06:54:17,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:17,337 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=38f136c511f939e3641e736608f6d72a, regionState=CLOSED 2023-02-15 06:54:17,337 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444057337"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444057337"}]},"ts":"1676444057337"} 2023-02-15 06:54:17,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-15 06:54:17,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 38f136c511f939e3641e736608f6d72a, server=jenkins-hbase4.apache.org,40475,1676444032771 in 164 msec 2023-02-15 06:54:17,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-15 06:54:17,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=38f136c511f939e3641e736608f6d72a, UNASSIGN in 174 msec 2023-02-15 06:54:17,342 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444057342"}]},"ts":"1676444057342"} 2023-02-15 06:54:17,344 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-15 06:54:17,346 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-15 06:54:17,347 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 188 msec 2023-02-15 06:54:17,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-15 06:54:17,464 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-15 06:54:17,464 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-15 06:54:17,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:54:17,467 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:54:17,468 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:54:17,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-15 06:54:17,472 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:17,472 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca 2023-02-15 06:54:17,472 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:17,474 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/recovered.edits] 2023-02-15 06:54:17,474 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/recovered.edits] 2023-02-15 06:54:17,474 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/recovered.edits] 2023-02-15 06:54:17,483 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/recovered.edits/4.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032/recovered.edits/4.seqid 2023-02-15 06:54:17,483 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/recovered.edits/7.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a/recovered.edits/7.seqid 2023-02-15 06:54:17,484 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/recovered.edits/7.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca/recovered.edits/7.seqid 2023-02-15 06:54:17,484 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/8938ff632984211f5fe6b773e2e60032 2023-02-15 06:54:17,484 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/38f136c511f939e3641e736608f6d72a 2023-02-15 06:54:17,485 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionNoStoreFile/83267c2682581a47fe295685701d02ca 2023-02-15 06:54:17,485 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-15 06:54:17,487 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:54:17,490 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-15 06:54:17,492 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-15 06:54:17,493 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:54:17,493 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-15 06:54:17,493 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444057493"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:17,493 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444057493"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:17,493 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444057493"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:17,495 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-15 06:54:17,495 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8938ff632984211f5fe6b773e2e60032, NAME => 'testSplitTableRegionNoStoreFile,,1676444055891.8938ff632984211f5fe6b773e2e60032.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 38f136c511f939e3641e736608f6d72a, NAME => 'testSplitTableRegionNoStoreFile,,1676444056393.38f136c511f939e3641e736608f6d72a.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 83267c2682581a47fe295685701d02ca, NAME => 'testSplitTableRegionNoStoreFile,41,1676444056393.83267c2682581a47fe295685701d02ca.', STARTKEY => '41', ENDKEY => ''}] 2023-02-15 06:54:17,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-15 06:54:17,495 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676444057495"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:17,497 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-15 06:54:17,499 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:54:17,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 35 msec 2023-02-15 06:54:17,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-15 06:54:17,569 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-15 06:54:17,589 INFO [Listener at localhost/37669] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=406 (was 397) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_951740873_15 at /127.0.0.1:36124 [Waiting for operation #50] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_951740873_15 at /127.0.0.1:49238 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=840 (was 836) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=249 (was 262), ProcessCount=168 (was 168), AvailableMemoryMB=3310 (was 3307) - AvailableMemoryMB LEAK? - 2023-02-15 06:54:17,606 INFO [Listener at localhost/37669] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=406, OpenFileDescriptor=840, MaxFileDescriptor=60000, SystemLoadAverage=249, ProcessCount=168, AvailableMemoryMB=3310 2023-02-15 06:54:17,606 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:17,609 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-15 06:54:17,610 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-15 06:54:17,612 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:17,613 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:54:17,614 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:54:17,615 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:17,616 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003 empty. 2023-02-15 06:54:17,616 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:17,617 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-15 06:54:17,634 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-15 06:54:17,635 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => a93cfa0b8140cbc18c2cd8f565cfd003, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:54:17,651 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:17,651 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing a93cfa0b8140cbc18c2cd8f565cfd003, disabling compactions & flushes 2023-02-15 06:54:17,651 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:17,651 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:17,651 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. after waiting 0 ms 2023-02-15 06:54:17,651 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:17,651 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:17,651 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:17,654 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:54:17,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676444057654"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444057654"}]},"ts":"1676444057654"} 2023-02-15 06:54:17,656 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:54:17,657 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:54:17,657 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444057657"}]},"ts":"1676444057657"} 2023-02-15 06:54:17,658 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-15 06:54:17,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:54:17,663 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:54:17,663 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:54:17,663 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:54:17,663 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:54:17,663 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a93cfa0b8140cbc18c2cd8f565cfd003, ASSIGN}] 2023-02-15 06:54:17,665 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a93cfa0b8140cbc18c2cd8f565cfd003, ASSIGN 2023-02-15 06:54:17,665 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a93cfa0b8140cbc18c2cd8f565cfd003, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40475,1676444032771; forceNewPlan=false, retain=false 2023-02-15 06:54:17,816 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:54:17,817 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=a93cfa0b8140cbc18c2cd8f565cfd003, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:17,817 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676444057817"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444057817"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444057817"}]},"ts":"1676444057817"} 2023-02-15 06:54:17,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure a93cfa0b8140cbc18c2cd8f565cfd003, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:17,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:17,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a93cfa0b8140cbc18c2cd8f565cfd003, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:54:17,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:17,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:17,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:17,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:17,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:17,975 INFO [StoreOpener-a93cfa0b8140cbc18c2cd8f565cfd003-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:17,977 DEBUG [StoreOpener-a93cfa0b8140cbc18c2cd8f565cfd003-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1 2023-02-15 06:54:17,977 DEBUG [StoreOpener-a93cfa0b8140cbc18c2cd8f565cfd003-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1 2023-02-15 06:54:17,977 INFO [StoreOpener-a93cfa0b8140cbc18c2cd8f565cfd003-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a93cfa0b8140cbc18c2cd8f565cfd003 columnFamilyName cf1 2023-02-15 06:54:17,978 INFO [StoreOpener-a93cfa0b8140cbc18c2cd8f565cfd003-1] regionserver.HStore(310): Store=a93cfa0b8140cbc18c2cd8f565cfd003/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:17,978 INFO [StoreOpener-a93cfa0b8140cbc18c2cd8f565cfd003-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:17,979 DEBUG [StoreOpener-a93cfa0b8140cbc18c2cd8f565cfd003-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2 2023-02-15 06:54:17,979 DEBUG [StoreOpener-a93cfa0b8140cbc18c2cd8f565cfd003-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2 2023-02-15 06:54:17,979 INFO [StoreOpener-a93cfa0b8140cbc18c2cd8f565cfd003-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a93cfa0b8140cbc18c2cd8f565cfd003 columnFamilyName cf2 2023-02-15 06:54:17,980 INFO [StoreOpener-a93cfa0b8140cbc18c2cd8f565cfd003-1] regionserver.HStore(310): Store=a93cfa0b8140cbc18c2cd8f565cfd003/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:17,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:17,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:17,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:17,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:17,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:54:17,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a93cfa0b8140cbc18c2cd8f565cfd003; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60579933, jitterRate=-0.09728865325450897}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:17,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:17,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003., pid=71, masterSystemTime=1676444057970 2023-02-15 06:54:17,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:17,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:17,989 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=a93cfa0b8140cbc18c2cd8f565cfd003, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:17,989 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676444057989"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444057989"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444057989"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444057989"}]},"ts":"1676444057989"} 2023-02-15 06:54:17,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-15 06:54:17,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure a93cfa0b8140cbc18c2cd8f565cfd003, server=jenkins-hbase4.apache.org,40475,1676444032771 in 173 msec 2023-02-15 06:54:17,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-15 06:54:17,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a93cfa0b8140cbc18c2cd8f565cfd003, ASSIGN in 329 msec 2023-02-15 06:54:17,996 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:54:17,996 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444057996"}]},"ts":"1676444057996"} 2023-02-15 06:54:17,997 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-15 06:54:17,999 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:54:18,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 388 msec 2023-02-15 06:54:18,120 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:18,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,122 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,122 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,122 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-15 06:54:18,122 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-15 06:54:18,122 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,122 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,124 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,124 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,124 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,124 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,124 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,124 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,125 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,125 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,126 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,126 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,126 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,126 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:18,126 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:18,126 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:18,126 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,126 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. started... 2023-02-15 06:54:18,126 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,126 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,126 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,126 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,127 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a93cfa0b8140cbc18c2cd8f565cfd003 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-15 06:54:18,127 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,127 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,127 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,129 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,129 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,130 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,130 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,130 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,130 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,137 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/3677f20b900f4f4aa7a6e200b103aa9d 2023-02-15 06:54:18,152 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/18df5422413049f5be21d718b03e8cc2 2023-02-15 06:54:18,156 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/3677f20b900f4f4aa7a6e200b103aa9d as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d 2023-02-15 06:54:18,159 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:54:18,160 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/18df5422413049f5be21d718b03e8cc2 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2 2023-02-15 06:54:18,164 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:54:18,165 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a93cfa0b8140cbc18c2cd8f565cfd003 in 38ms, sequenceid=6, compaction requested=false 2023-02-15 06:54:18,165 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,165 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:18,166 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:18,166 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:18,166 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:18,166 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,166 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,166 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,167 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,167 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,168 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,168 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-15 06:54:18,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1aec5b8e[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:18,170 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-15 06:54:18,170 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,171 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,171 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,171 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,171 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,171 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:18,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,171 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,171 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,172 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:18,171 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,172 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,172 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,172 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,172 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:18,172 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:18,173 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,173 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,173 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,177 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,177 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,177 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,177 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,177 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,177 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,177 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:54:18,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:54:18,180 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-15 06:54:18,180 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:18,180 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:18,180 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,180 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:18,181 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,181 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,181 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,181 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,181 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,182 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,182 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,182 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,183 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,183 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,185 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,185 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,185 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,192 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,192 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:18,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,193 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:18,193 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,192 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,192 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,193 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,193 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:18,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,193 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,193 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,193 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,194 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,194 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,194 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,194 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,293 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:18,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:18,309 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:18,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:18,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,311 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,311 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,311 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-15 06:54:18,311 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-15 06:54:18,312 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,312 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,318 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,318 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,318 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,318 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,318 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,319 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:18,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,320 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,320 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,320 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,320 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,320 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,320 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,320 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,320 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,321 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,320 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,321 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,320 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,321 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,321 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,321 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,321 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,321 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,322 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,323 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,323 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,323 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,323 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,323 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,324 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:18,324 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:18,324 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:18,324 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. started... 2023-02-15 06:54:18,324 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a93cfa0b8140cbc18c2cd8f565cfd003 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:18,340 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/c595e942710147dab770ee17b56584be 2023-02-15 06:54:18,357 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/c0212b1c94184df2a46581dbd45b6c83 2023-02-15 06:54:18,362 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/c595e942710147dab770ee17b56584be as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be 2023-02-15 06:54:18,367 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:54:18,367 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/c0212b1c94184df2a46581dbd45b6c83 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83 2023-02-15 06:54:18,372 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:54:18,372 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a93cfa0b8140cbc18c2cd8f565cfd003 in 48ms, sequenceid=19, compaction requested=false 2023-02-15 06:54:18,373 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:18,373 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:18,374 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:18,374 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:18,374 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,374 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,374 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,376 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,376 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,376 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,376 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,378 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-15 06:54:18,378 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27c118f0[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:18,378 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-15 06:54:18,378 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,380 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,380 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,380 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,381 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,381 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:18,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,381 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,381 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,381 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,381 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:18,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,381 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,381 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,381 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:18,381 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,381 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:18,383 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,383 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,383 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,384 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,384 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,384 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,384 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,384 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,384 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,384 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:54:18,387 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:54:18,387 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-15 06:54:18,388 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:18,388 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:18,388 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,388 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:18,389 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,389 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,390 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,389 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,390 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,389 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,390 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,390 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,391 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,391 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,395 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,396 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,396 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,400 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,400 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,400 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,400 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,400 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,400 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,401 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,401 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:18,401 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,400 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,401 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,400 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,401 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,401 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,401 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,401 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,401 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:18,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,404 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:18,404 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,404 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,404 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,404 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,504 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:18,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:18,519 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:18,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:18,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,521 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,521 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,521 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-15 06:54:18,521 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-15 06:54:18,521 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,521 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,523 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,523 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,523 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,523 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,523 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,523 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:18,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,525 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,525 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,525 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,525 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,525 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,525 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,525 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,525 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. started... 2023-02-15 06:54:18,526 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a93cfa0b8140cbc18c2cd8f565cfd003 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:18,527 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,528 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,528 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,528 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,529 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,529 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,540 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/9c9b9f0c05974a31859444e3450d35ae 2023-02-15 06:54:18,556 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/792fe3b1080e4a34a94b4abb457adf71 2023-02-15 06:54:18,560 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/9c9b9f0c05974a31859444e3450d35ae as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae 2023-02-15 06:54:18,565 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:54:18,565 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/792fe3b1080e4a34a94b4abb457adf71 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71 2023-02-15 06:54:18,570 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:54:18,571 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a93cfa0b8140cbc18c2cd8f565cfd003 in 45ms, sequenceid=32, compaction requested=true 2023-02-15 06:54:18,571 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:18,571 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:18,571 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:18,571 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:18,571 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,571 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,571 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,577 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,577 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,577 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,577 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-15 06:54:18,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@496e4f36[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:18,579 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-15 06:54:18,579 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,580 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,580 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,580 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,580 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:18,580 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,581 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:18,580 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,581 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:18,581 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,581 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,581 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:18,582 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,582 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,582 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,583 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,583 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,583 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,583 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,583 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,583 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,583 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:54:18,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:54:18,586 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-15 06:54:18,586 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:18,586 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:18,586 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,586 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:18,589 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,589 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,589 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,589 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,589 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,589 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,589 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,589 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,590 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,590 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,592 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,592 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,592 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,595 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,595 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:18,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,595 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,596 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:18,596 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,596 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:18,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,596 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,596 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,596 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,596 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,597 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,597 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,696 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:18,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:18,711 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:18,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:18,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,712 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,712 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,712 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-15 06:54:18,712 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-15 06:54:18,713 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,713 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,714 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,714 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,714 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,714 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,714 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,714 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:18,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,715 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,716 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,716 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,716 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,716 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,716 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:18,716 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:18,716 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:18,716 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. started... 2023-02-15 06:54:18,716 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a93cfa0b8140cbc18c2cd8f565cfd003 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:18,718 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,718 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,718 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,718 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,718 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,718 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,728 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/5099df7157024afaa6d2084ad018b5ce 2023-02-15 06:54:18,740 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/441c7b821ac741aa83d0c66b64613bc5 2023-02-15 06:54:18,744 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/5099df7157024afaa6d2084ad018b5ce as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5099df7157024afaa6d2084ad018b5ce 2023-02-15 06:54:18,748 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5099df7157024afaa6d2084ad018b5ce, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:54:18,749 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/441c7b821ac741aa83d0c66b64613bc5 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/441c7b821ac741aa83d0c66b64613bc5 2023-02-15 06:54:18,752 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/441c7b821ac741aa83d0c66b64613bc5, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:54:18,753 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a93cfa0b8140cbc18c2cd8f565cfd003 in 37ms, sequenceid=45, compaction requested=true 2023-02-15 06:54:18,753 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:18,753 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:18,753 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:18,753 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:18,753 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,753 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,754 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,756 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,756 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,756 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,756 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-15 06:54:18,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d5bd43e[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:18,758 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-15 06:54:18,758 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,760 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,760 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,760 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,760 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,760 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,760 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:18,760 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,760 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:18,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,760 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,761 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,760 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,761 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:18,761 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,761 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:18,762 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,762 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,762 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,762 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,763 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,763 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,763 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,763 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,764 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,764 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:54:18,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:54:18,766 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-15 06:54:18,766 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:18,766 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:18,766 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,766 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:18,772 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,772 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,772 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,772 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,773 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,773 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,775 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,775 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,775 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,778 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,778 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,778 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:18,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,778 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,779 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,779 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,779 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,779 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,779 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,779 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,879 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:18,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:18,892 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:18,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:18,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,893 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,893 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,893 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-15 06:54:18,893 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-15 06:54:18,893 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,893 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,895 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,895 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,895 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,895 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,895 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,895 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:18,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,896 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,896 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,896 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,896 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,897 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,897 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,897 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,897 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:18,897 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:18,897 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:18,897 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. started... 2023-02-15 06:54:18,897 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a93cfa0b8140cbc18c2cd8f565cfd003 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:18,899 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,900 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,900 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,900 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,900 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,900 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,907 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/3164a2aa76b44799aebc4ab7b85cefaa 2023-02-15 06:54:18,919 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/8ec47342b8a34ce6ba11286bdf7eb268 2023-02-15 06:54:18,923 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/3164a2aa76b44799aebc4ab7b85cefaa as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3164a2aa76b44799aebc4ab7b85cefaa 2023-02-15 06:54:18,928 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3164a2aa76b44799aebc4ab7b85cefaa, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:54:18,929 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/8ec47342b8a34ce6ba11286bdf7eb268 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/8ec47342b8a34ce6ba11286bdf7eb268 2023-02-15 06:54:18,933 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/8ec47342b8a34ce6ba11286bdf7eb268, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:54:18,934 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a93cfa0b8140cbc18c2cd8f565cfd003 in 37ms, sequenceid=58, compaction requested=true 2023-02-15 06:54:18,934 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:18,934 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:18,934 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:18,934 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:18,934 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,934 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:18,934 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:18,937 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,937 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,937 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,937 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:18,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-15 06:54:18,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@513d030[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:18,940 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-15 06:54:18,940 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,941 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,941 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,941 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,941 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,941 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,941 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,941 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:18,941 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,942 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:18,941 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:18,941 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,942 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:18,942 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:18,942 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:18,944 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,944 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,944 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,945 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,945 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:18,945 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,945 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,945 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,945 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,945 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:18,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:54:18,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:54:18,948 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-15 06:54:18,949 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:18,949 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:18,949 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,949 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:18,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,951 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,950 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,951 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:18,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:18,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,951 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:18,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,952 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,952 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:18,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,954 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:18,954 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,955 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,957 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,958 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:18,958 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,958 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:18,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:18,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,958 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:18,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,959 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:18,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:18,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:18,959 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,959 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:18,959 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:18,959 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,959 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:18,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,059 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:19,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:19,072 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:19,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:19,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,074 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:19,074 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,074 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-15 06:54:19,074 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-15 06:54:19,074 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,074 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,079 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,079 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,079 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,079 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,080 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,080 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:19,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,081 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,081 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,081 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,081 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,081 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,082 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,083 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,083 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:19,083 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:19,083 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,083 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,083 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,083 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,083 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,083 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,083 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. started... 2023-02-15 06:54:19,083 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,083 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,083 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a93cfa0b8140cbc18c2cd8f565cfd003 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:54:19,083 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,083 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,085 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,085 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,085 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,099 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/12535f15786c43b88367bcbc1f01d0da 2023-02-15 06:54:19,115 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/4781241327b641f488ad79fc8157b5f3 2023-02-15 06:54:19,120 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/12535f15786c43b88367bcbc1f01d0da as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/12535f15786c43b88367bcbc1f01d0da 2023-02-15 06:54:19,124 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/12535f15786c43b88367bcbc1f01d0da, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:54:19,125 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/4781241327b641f488ad79fc8157b5f3 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/4781241327b641f488ad79fc8157b5f3 2023-02-15 06:54:19,130 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/4781241327b641f488ad79fc8157b5f3, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:54:19,131 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a93cfa0b8140cbc18c2cd8f565cfd003 in 48ms, sequenceid=71, compaction requested=true 2023-02-15 06:54:19,131 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:19,131 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,131 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:19,131 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:19,131 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,131 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,132 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,134 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,134 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,134 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,134 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-15 06:54:19,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a3a5efa[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:19,137 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-15 06:54:19,137 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,138 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,138 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,138 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,138 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,138 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,138 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:19,138 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,139 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,139 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:19,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,139 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,139 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:19,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,139 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,139 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,139 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:19,140 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,140 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,140 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,140 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,141 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,141 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,141 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,142 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,142 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,142 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:54:19,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:54:19,146 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-15 06:54:19,146 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:19,146 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:19,146 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,146 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:19,147 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,147 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,147 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,147 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,147 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,147 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,147 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,148 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,149 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,149 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,151 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,151 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,151 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,154 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,154 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,155 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,154 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,155 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:19,155 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,155 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:19,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,256 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:19,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:19,271 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:19,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:19,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,273 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:19,273 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,273 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-15 06:54:19,273 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-15 06:54:19,273 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,274 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,275 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,275 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,275 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,275 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,275 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,275 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:19,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,276 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,277 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,277 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,277 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,277 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,277 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:19,277 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,277 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,277 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. started... 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,277 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,277 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a93cfa0b8140cbc18c2cd8f565cfd003 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-15 06:54:19,280 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,280 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,281 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,281 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,281 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,281 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,290 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/2196511bb07747c585a68b86f2d1b102 2023-02-15 06:54:19,294 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2196511bb07747c585a68b86f2d1b102 2023-02-15 06:54:19,303 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/727bdd9c8dbe4c31a84af2fa728250b1 2023-02-15 06:54:19,306 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 727bdd9c8dbe4c31a84af2fa728250b1 2023-02-15 06:54:19,308 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/2196511bb07747c585a68b86f2d1b102 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102 2023-02-15 06:54:19,311 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2196511bb07747c585a68b86f2d1b102 2023-02-15 06:54:19,311 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102, entries=9, sequenceid=83, filesize=5.1 K 2023-02-15 06:54:19,312 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/727bdd9c8dbe4c31a84af2fa728250b1 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1 2023-02-15 06:54:19,316 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 727bdd9c8dbe4c31a84af2fa728250b1 2023-02-15 06:54:19,316 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1, entries=9, sequenceid=83, filesize=5.1 K 2023-02-15 06:54:19,316 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for a93cfa0b8140cbc18c2cd8f565cfd003 in 39ms, sequenceid=83, compaction requested=true 2023-02-15 06:54:19,317 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:19,317 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,317 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:19,317 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:19,317 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,317 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,317 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,319 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,319 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,319 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,319 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-15 06:54:19,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1196a374[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:19,321 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-15 06:54:19,321 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,322 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,322 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,322 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,323 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,323 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,323 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:19,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,323 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,323 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,323 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,323 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,323 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:19,323 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,323 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:19,323 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:19,332 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,332 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,332 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,332 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,333 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,333 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,333 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,333 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,333 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,333 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:54:19,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:54:19,337 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-15 06:54:19,337 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:19,337 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:19,337 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,337 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:19,338 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,338 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,338 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,339 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,339 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,339 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,339 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,340 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,340 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,341 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,342 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,342 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,345 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,345 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,346 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:19,346 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,346 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:19,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,347 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,347 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,347 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,347 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,347 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,447 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:19,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:19,455 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:19,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:19,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,457 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:19,457 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,457 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-15 06:54:19,457 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-15 06:54:19,457 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,457 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,458 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,459 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,458 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,458 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,459 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,459 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:19,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,459 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,460 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,460 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,460 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,460 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,460 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,461 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,461 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:19,461 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,461 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:19,461 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. started... 2023-02-15 06:54:19,461 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a93cfa0b8140cbc18c2cd8f565cfd003 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-15 06:54:19,463 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,463 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,463 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,463 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,463 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,463 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,470 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/d9921529217e4617b85a44197272bd7e 2023-02-15 06:54:19,474 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9921529217e4617b85a44197272bd7e 2023-02-15 06:54:19,481 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/bece57e3ef26499cbd533c0591ba291d 2023-02-15 06:54:19,485 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bece57e3ef26499cbd533c0591ba291d 2023-02-15 06:54:19,486 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/d9921529217e4617b85a44197272bd7e as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/d9921529217e4617b85a44197272bd7e 2023-02-15 06:54:19,490 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9921529217e4617b85a44197272bd7e 2023-02-15 06:54:19,490 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/d9921529217e4617b85a44197272bd7e, entries=5, sequenceid=91, filesize=5.0 K 2023-02-15 06:54:19,491 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/bece57e3ef26499cbd533c0591ba291d as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/bece57e3ef26499cbd533c0591ba291d 2023-02-15 06:54:19,495 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bece57e3ef26499cbd533c0591ba291d 2023-02-15 06:54:19,495 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/bece57e3ef26499cbd533c0591ba291d, entries=5, sequenceid=91, filesize=5.0 K 2023-02-15 06:54:19,496 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for a93cfa0b8140cbc18c2cd8f565cfd003 in 35ms, sequenceid=91, compaction requested=true 2023-02-15 06:54:19,496 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:19,496 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,496 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:19,496 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:19,496 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,496 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,496 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,498 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,498 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,498 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,498 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,500 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-15 06:54:19,500 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@324f7f5c[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:19,500 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-15 06:54:19,500 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,502 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,502 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,502 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:19,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,502 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,502 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,502 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,502 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,502 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:19,502 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:19,502 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,502 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,502 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:19,503 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,503 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,504 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,504 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,504 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,504 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,504 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,504 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,504 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,504 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:54:19,507 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:54:19,507 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-15 06:54:19,507 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:19,507 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:19,508 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,508 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:19,509 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,509 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,509 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,509 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,509 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,509 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,509 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,510 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,510 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,511 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,514 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,514 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,514 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,522 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,522 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:19,522 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,522 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,523 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,523 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:19,523 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,523 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,523 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,623 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:19,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:19,632 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:54:19,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:54:19,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,634 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:19,634 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,634 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-15 06:54:19,634 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-15 06:54:19,634 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,634 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,636 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,636 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,636 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,636 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,636 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,636 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:54:19,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,637 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,637 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-15 06:54:19,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:54:19,638 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:54:19,638 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,638 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. started... 2023-02-15 06:54:19,639 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a93cfa0b8140cbc18c2cd8f565cfd003 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-15 06:54:19,639 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,640 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,640 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,640 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,641 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,641 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,656 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/0f5666bd44054b28b040f250412c0736 2023-02-15 06:54:19,660 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f5666bd44054b28b040f250412c0736 2023-02-15 06:54:19,674 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/5ce24cc23e9c422c9f2f96980442e4a9 2023-02-15 06:54:19,678 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ce24cc23e9c422c9f2f96980442e4a9 2023-02-15 06:54:19,679 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/0f5666bd44054b28b040f250412c0736 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/0f5666bd44054b28b040f250412c0736 2023-02-15 06:54:19,683 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f5666bd44054b28b040f250412c0736 2023-02-15 06:54:19,683 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/0f5666bd44054b28b040f250412c0736, entries=5, sequenceid=99, filesize=5.0 K 2023-02-15 06:54:19,684 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/5ce24cc23e9c422c9f2f96980442e4a9 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/5ce24cc23e9c422c9f2f96980442e4a9 2023-02-15 06:54:19,687 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ce24cc23e9c422c9f2f96980442e4a9 2023-02-15 06:54:19,687 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/5ce24cc23e9c422c9f2f96980442e4a9, entries=5, sequenceid=99, filesize=5.0 K 2023-02-15 06:54:19,688 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for a93cfa0b8140cbc18c2cd8f565cfd003 in 49ms, sequenceid=99, compaction requested=true 2023-02-15 06:54:19,688 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:19,688 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,688 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:54:19,688 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:54:19,689 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,689 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-15 06:54:19,689 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-15 06:54:19,691 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,691 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,692 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,692 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:54:19,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-15 06:54:19,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29c6bb75[Count = 0] remaining members to acquire global barrier 2023-02-15 06:54:19,693 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-15 06:54:19,693 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,695 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,695 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,695 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:54:19,695 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,695 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,695 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-15 06:54:19,695 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,695 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:54:19,695 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-15 06:54:19,695 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:54:19,695 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:54:19,697 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,697 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,697 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,698 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,698 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,698 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,698 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-15 06:54:19,698 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,698 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,698 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-15 06:54:19,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:54:19,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:54:19,701 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-15 06:54:19,701 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:54:19,701 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:54:19,701 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,701 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:54:19,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,703 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:54:19,703 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,704 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,705 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:54:19,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:54:19,706 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,706 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,707 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,709 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:54:19,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:54:19,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,709 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-15 06:54:19,710 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:54:19,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:54:19,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,710 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:19,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,710 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:54:19,710 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,710 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:54:19,710 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:54:19,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:54:19,710 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:19,710 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,710 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:19,810 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:54:19,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:54:19,814 DEBUG [Listener at localhost/37669] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,815 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:19,816 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:19,816 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:19,816 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:19,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a93cfa0b8140cbc18c2cd8f565cfd003, UNASSIGN}] 2023-02-15 06:54:19,818 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a93cfa0b8140cbc18c2cd8f565cfd003, UNASSIGN 2023-02-15 06:54:19,819 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=a93cfa0b8140cbc18c2cd8f565cfd003, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:19,819 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676444059819"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444059819"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444059819"}]},"ts":"1676444059819"} 2023-02-15 06:54:19,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure a93cfa0b8140cbc18c2cd8f565cfd003, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:19,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:19,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a93cfa0b8140cbc18c2cd8f565cfd003, disabling compactions & flushes 2023-02-15 06:54:19,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. after waiting 0 ms 2023-02-15 06:54:19,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:19,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing a93cfa0b8140cbc18c2cd8f565cfd003 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-15 06:54:19,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/5cad67575de04fd3b9952f726b573ffc 2023-02-15 06:54:19,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5cad67575de04fd3b9952f726b573ffc 2023-02-15 06:54:19,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/6b6c1c2ca40a49cc8b801e45ce4dea2b 2023-02-15 06:54:19,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b6c1c2ca40a49cc8b801e45ce4dea2b 2023-02-15 06:54:19,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf1/5cad67575de04fd3b9952f726b573ffc as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5cad67575de04fd3b9952f726b573ffc 2023-02-15 06:54:20,001 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5cad67575de04fd3b9952f726b573ffc 2023-02-15 06:54:20,001 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5cad67575de04fd3b9952f726b573ffc, entries=1, sequenceid=103, filesize=4.9 K 2023-02-15 06:54:20,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.tmp/cf2/6b6c1c2ca40a49cc8b801e45ce4dea2b as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/6b6c1c2ca40a49cc8b801e45ce4dea2b 2023-02-15 06:54:20,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b6c1c2ca40a49cc8b801e45ce4dea2b 2023-02-15 06:54:20,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/6b6c1c2ca40a49cc8b801e45ce4dea2b, entries=1, sequenceid=103, filesize=4.9 K 2023-02-15 06:54:20,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for a93cfa0b8140cbc18c2cd8f565cfd003 in 35ms, sequenceid=103, compaction requested=true 2023-02-15 06:54:20,011 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-15 06:54:20,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:20,012 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003. 2023-02-15 06:54:20,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a93cfa0b8140cbc18c2cd8f565cfd003: 2023-02-15 06:54:20,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,014 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=a93cfa0b8140cbc18c2cd8f565cfd003, regionState=CLOSED 2023-02-15 06:54:20,014 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676444060014"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444060014"}]},"ts":"1676444060014"} 2023-02-15 06:54:20,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-15 06:54:20,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure a93cfa0b8140cbc18c2cd8f565cfd003, server=jenkins-hbase4.apache.org,40475,1676444032771 in 195 msec 2023-02-15 06:54:20,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-15 06:54:20,019 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=a93cfa0b8140cbc18c2cd8f565cfd003, UNASSIGN in 199 msec 2023-02-15 06:54:20,034 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=a93cfa0b8140cbc18c2cd8f565cfd003, threads=16 2023-02-15 06:54:20,035 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,035 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/441c7b821ac741aa83d0c66b64613bc5 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,035 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/4781241327b641f488ad79fc8157b5f3 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,035 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/5ce24cc23e9c422c9f2f96980442e4a9 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,038 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/6b6c1c2ca40a49cc8b801e45ce4dea2b for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,039 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,039 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,039 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/8ec47342b8a34ce6ba11286bdf7eb268 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,039 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/bece57e3ef26499cbd533c0591ba291d for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,040 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,044 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/0f5666bd44054b28b040f250412c0736 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,044 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/12535f15786c43b88367bcbc1f01d0da for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,044 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,044 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3164a2aa76b44799aebc4ab7b85cefaa for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,045 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,048 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5099df7157024afaa6d2084ad018b5ce for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,051 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/441c7b821ac741aa83d0c66b64613bc5, top=false 2023-02-15 06:54:20,054 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ce24cc23e9c422c9f2f96980442e4a9 2023-02-15 06:54:20,054 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/12535f15786c43b88367bcbc1f01d0da, top=false 2023-02-15 06:54:20,055 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bece57e3ef26499cbd533c0591ba291d 2023-02-15 06:54:20,056 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/4781241327b641f488ad79fc8157b5f3, top=false 2023-02-15 06:54:20,058 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5099df7157024afaa6d2084ad018b5ce, top=false 2023-02-15 06:54:20,060 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/8ec47342b8a34ce6ba11286bdf7eb268, top=false 2023-02-15 06:54:20,061 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2196511bb07747c585a68b86f2d1b102 2023-02-15 06:54:20,061 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f5666bd44054b28b040f250412c0736 2023-02-15 06:54:20,062 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3164a2aa76b44799aebc4ab7b85cefaa, top=false 2023-02-15 06:54:20,064 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b6c1c2ca40a49cc8b801e45ce4dea2b 2023-02-15 06:54:20,066 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ce24cc23e9c422c9f2f96980442e4a9 2023-02-15 06:54:20,066 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/5ce24cc23e9c422c9f2f96980442e4a9, top=true 2023-02-15 06:54:20,068 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f5666bd44054b28b040f250412c0736 2023-02-15 06:54:20,069 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/0f5666bd44054b28b040f250412c0736, top=true 2023-02-15 06:54:20,072 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bece57e3ef26499cbd533c0591ba291d 2023-02-15 06:54:20,072 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/bece57e3ef26499cbd533c0591ba291d, top=true 2023-02-15 06:54:20,073 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 727bdd9c8dbe4c31a84af2fa728250b1 2023-02-15 06:54:20,082 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b6c1c2ca40a49cc8b801e45ce4dea2b 2023-02-15 06:54:20,082 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/6b6c1c2ca40a49cc8b801e45ce4dea2b, top=true 2023-02-15 06:54:20,100 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5ce24cc23e9c422c9f2f96980442e4a9 for child: e51419864fae0902e5f91adf742ee434, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,100 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-4781241327b641f488ad79fc8157b5f3 for child: 234d84c9a604cb1d5f1618efa32bcb73, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,100 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/5ce24cc23e9c422c9f2f96980442e4a9 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,100 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5cad67575de04fd3b9952f726b573ffc for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,101 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-8ec47342b8a34ce6ba11286bdf7eb268 for child: 234d84c9a604cb1d5f1618efa32bcb73, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,101 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-3164a2aa76b44799aebc4ab7b85cefaa for child: 234d84c9a604cb1d5f1618efa32bcb73, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,104 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-0f5666bd44054b28b040f250412c0736 for child: e51419864fae0902e5f91adf742ee434, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,104 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/0f5666bd44054b28b040f250412c0736 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,104 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,108 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-12535f15786c43b88367bcbc1f01d0da for child: 234d84c9a604cb1d5f1618efa32bcb73, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,112 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/8ec47342b8a34ce6ba11286bdf7eb268 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,112 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,112 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/4781241327b641f488ad79fc8157b5f3 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,112 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/d9921529217e4617b85a44197272bd7e for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,113 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3164a2aa76b44799aebc4ab7b85cefaa for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,116 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9921529217e4617b85a44197272bd7e 2023-02-15 06:54:20,116 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-441c7b821ac741aa83d0c66b64613bc5 for child: 234d84c9a604cb1d5f1618efa32bcb73, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,117 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5cad67575de04fd3b9952f726b573ffc 2023-02-15 06:54:20,117 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/12535f15786c43b88367bcbc1f01d0da for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,120 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/441c7b821ac741aa83d0c66b64613bc5 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,121 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9921529217e4617b85a44197272bd7e 2023-02-15 06:54:20,121 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/d9921529217e4617b85a44197272bd7e, top=true 2023-02-15 06:54:20,121 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5cad67575de04fd3b9952f726b573ffc 2023-02-15 06:54:20,121 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5cad67575de04fd3b9952f726b573ffc, top=true 2023-02-15 06:54:20,124 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-6b6c1c2ca40a49cc8b801e45ce4dea2b for child: e51419864fae0902e5f91adf742ee434, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,124 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/6b6c1c2ca40a49cc8b801e45ce4dea2b for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,144 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-bece57e3ef26499cbd533c0591ba291d for child: e51419864fae0902e5f91adf742ee434, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,144 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/bece57e3ef26499cbd533c0591ba291d for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,156 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5099df7157024afaa6d2084ad018b5ce for child: 234d84c9a604cb1d5f1618efa32bcb73, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,162 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5099df7157024afaa6d2084ad018b5ce for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,176 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5cad67575de04fd3b9952f726b573ffc for child: e51419864fae0902e5f91adf742ee434, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,176 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5cad67575de04fd3b9952f726b573ffc for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,177 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/.splits/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-d9921529217e4617b85a44197272bd7e for child: e51419864fae0902e5f91adf742ee434, parent: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,177 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/d9921529217e4617b85a44197272bd7e for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,182 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 727bdd9c8dbe4c31a84af2fa728250b1 2023-02-15 06:54:20,183 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2196511bb07747c585a68b86f2d1b102 2023-02-15 06:54:20,200 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,235 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,248 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,261 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,262 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,268 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,271 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,271 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1 for region: a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,271 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region a93cfa0b8140cbc18c2cd8f565cfd003 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-15 06:54:20,320 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-15 06:54:20,322 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-15 06:54:20,324 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676444060323"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676444060323"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676444060323"}]},"ts":"1676444060323"} 2023-02-15 06:54:20,324 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676444060323"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444060323"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444060323"}]},"ts":"1676444060323"} 2023-02-15 06:54:20,324 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676444060323"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444060323"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444060323"}]},"ts":"1676444060323"} 2023-02-15 06:54:20,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=234d84c9a604cb1d5f1618efa32bcb73, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e51419864fae0902e5f91adf742ee434, ASSIGN}] 2023-02-15 06:54:20,328 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e51419864fae0902e5f91adf742ee434, ASSIGN 2023-02-15 06:54:20,329 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e51419864fae0902e5f91adf742ee434, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40475,1676444032771; forceNewPlan=false, retain=false 2023-02-15 06:54:20,330 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=234d84c9a604cb1d5f1618efa32bcb73, ASSIGN 2023-02-15 06:54:20,331 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=234d84c9a604cb1d5f1618efa32bcb73, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40475,1676444032771; forceNewPlan=false, retain=false 2023-02-15 06:54:20,480 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-15 06:54:20,482 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e51419864fae0902e5f91adf742ee434, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:20,482 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676444060482"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444060482"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444060482"}]},"ts":"1676444060482"} 2023-02-15 06:54:20,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure e51419864fae0902e5f91adf742ee434, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:20,486 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=234d84c9a604cb1d5f1618efa32bcb73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:20,486 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676444060486"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444060486"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444060486"}]},"ts":"1676444060486"} 2023-02-15 06:54:20,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure 234d84c9a604cb1d5f1618efa32bcb73, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:20,505 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-15 06:54:20,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:20,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 234d84c9a604cb1d5f1618efa32bcb73, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.', STARTKEY => '', ENDKEY => '60'} 2023-02-15 06:54:20,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:20,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:20,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:20,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:20,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:20,640 INFO [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:20,641 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1 2023-02-15 06:54:20,641 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1 2023-02-15 06:54:20,641 INFO [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 234d84c9a604cb1d5f1618efa32bcb73 columnFamilyName cf1 2023-02-15 06:54:20,653 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,655 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102-bottom 2023-02-15 06:54:20,660 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d-bottom 2023-02-15 06:54:20,664 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae-bottom 2023-02-15 06:54:20,669 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be-bottom 2023-02-15 06:54:20,673 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-12535f15786c43b88367bcbc1f01d0da 2023-02-15 06:54:20,676 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-3164a2aa76b44799aebc4ab7b85cefaa 2023-02-15 06:54:20,680 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5099df7157024afaa6d2084ad018b5ce 2023-02-15 06:54:20,680 INFO [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(310): Store=234d84c9a604cb1d5f1618efa32bcb73/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:20,680 INFO [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:20,681 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2 2023-02-15 06:54:20,681 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2 2023-02-15 06:54:20,682 INFO [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 234d84c9a604cb1d5f1618efa32bcb73 columnFamilyName cf2 2023-02-15 06:54:20,695 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2-bottom 2023-02-15 06:54:20,698 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,700 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1-bottom 2023-02-15 06:54:20,705 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71-bottom 2023-02-15 06:54:20,710 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83-bottom 2023-02-15 06:54:20,713 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-441c7b821ac741aa83d0c66b64613bc5 2023-02-15 06:54:20,717 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-4781241327b641f488ad79fc8157b5f3 2023-02-15 06:54:20,720 DEBUG [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-8ec47342b8a34ce6ba11286bdf7eb268 2023-02-15 06:54:20,721 INFO [StoreOpener-234d84c9a604cb1d5f1618efa32bcb73-1] regionserver.HStore(310): Store=234d84c9a604cb1d5f1618efa32bcb73/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:20,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:20,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:20,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:20,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:20,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 234d84c9a604cb1d5f1618efa32bcb73; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68659472, jitterRate=0.023105859756469727}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:20,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 234d84c9a604cb1d5f1618efa32bcb73: 2023-02-15 06:54:20,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73., pid=78, masterSystemTime=1676444060635 2023-02-15 06:54:20,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:20,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:54:20,731 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:54:20,731 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:54:20,731 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:20,731 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1912): 234d84c9a604cb1d5f1618efa32bcb73/cf1 is initiating minor compaction (all files) 2023-02-15 06:54:20,731 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:20,731 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1912): 234d84c9a604cb1d5f1618efa32bcb73/cf2 is initiating minor compaction (all files) 2023-02-15 06:54:20,732 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 234d84c9a604cb1d5f1618efa32bcb73/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:20,732 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 234d84c9a604cb1d5f1618efa32bcb73/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:20,732 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5099df7157024afaa6d2084ad018b5ce, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-3164a2aa76b44799aebc4ab7b85cefaa, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-12535f15786c43b88367bcbc1f01d0da, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102-bottom] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/.tmp, totalSize=35.0 K 2023-02-15 06:54:20,732 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-441c7b821ac741aa83d0c66b64613bc5, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-8ec47342b8a34ce6ba11286bdf7eb268, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-4781241327b641f488ad79fc8157b5f3, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1-bottom] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/.tmp, totalSize=35.0 K 2023-02-15 06:54:20,732 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444058115 2023-02-15 06:54:20,733 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444058115 2023-02-15 06:54:20,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:20,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:20,733 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444058295 2023-02-15 06:54:20,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:20,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e51419864fae0902e5f91adf742ee434, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.', STARTKEY => '60', ENDKEY => ''} 2023-02-15 06:54:20,733 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444058506 2023-02-15 06:54:20,734 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=234d84c9a604cb1d5f1618efa32bcb73, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:20,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:20,734 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676444060734"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444060734"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444060734"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444060734"}]},"ts":"1676444060734"} 2023-02-15 06:54:20,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:20,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:20,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:20,734 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444058295 2023-02-15 06:54:20,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:20,735 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5099df7157024afaa6d2084ad018b5ce, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444058698 2023-02-15 06:54:20,735 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444058506 2023-02-15 06:54:20,736 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-441c7b821ac741aa83d0c66b64613bc5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444058698 2023-02-15 06:54:20,736 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-3164a2aa76b44799aebc4ab7b85cefaa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444058880 2023-02-15 06:54:20,737 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-12535f15786c43b88367bcbc1f01d0da, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444059061 2023-02-15 06:54:20,737 INFO [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:20,737 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-8ec47342b8a34ce6ba11286bdf7eb268, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444058880 2023-02-15 06:54:20,737 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676444059258 2023-02-15 06:54:20,737 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-4781241327b641f488ad79fc8157b5f3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444059061 2023-02-15 06:54:20,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-02-15 06:54:20,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure 234d84c9a604cb1d5f1618efa32bcb73, server=jenkins-hbase4.apache.org,40475,1676444032771 in 248 msec 2023-02-15 06:54:20,738 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676444059258 2023-02-15 06:54:20,739 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1 2023-02-15 06:54:20,740 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1 2023-02-15 06:54:20,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=234d84c9a604cb1d5f1618efa32bcb73, ASSIGN in 411 msec 2023-02-15 06:54:20,740 INFO [StoreOpener-e51419864fae0902e5f91adf742ee434-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e51419864fae0902e5f91adf742ee434 columnFamilyName cf1 2023-02-15 06:54:20,750 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,751 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102-top 2023-02-15 06:54:20,755 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] throttle.PressureAwareThroughputController(145): 234d84c9a604cb1d5f1618efa32bcb73#cf1#compaction#72 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:20,755 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] throttle.PressureAwareThroughputController(145): 234d84c9a604cb1d5f1618efa32bcb73#cf2#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:20,757 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d-top 2023-02-15 06:54:20,761 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae-top 2023-02-15 06:54:20,764 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be-top 2023-02-15 06:54:20,779 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f5666bd44054b28b040f250412c0736 2023-02-15 06:54:20,779 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-0f5666bd44054b28b040f250412c0736 2023-02-15 06:54:20,783 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5cad67575de04fd3b9952f726b573ffc 2023-02-15 06:54:20,783 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5cad67575de04fd3b9952f726b573ffc 2023-02-15 06:54:20,785 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/.tmp/cf1/17324af8340241d2b493da2644597f77 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/17324af8340241d2b493da2644597f77 2023-02-15 06:54:20,787 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d9921529217e4617b85a44197272bd7e 2023-02-15 06:54:20,787 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-d9921529217e4617b85a44197272bd7e 2023-02-15 06:54:20,787 INFO [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(310): Store=e51419864fae0902e5f91adf742ee434/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:20,787 INFO [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:20,788 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2 2023-02-15 06:54:20,788 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2 2023-02-15 06:54:20,789 INFO [StoreOpener-e51419864fae0902e5f91adf742ee434-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e51419864fae0902e5f91adf742ee434 columnFamilyName cf2 2023-02-15 06:54:20,790 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 234d84c9a604cb1d5f1618efa32bcb73/cf1 of 234d84c9a604cb1d5f1618efa32bcb73 into 17324af8340241d2b493da2644597f77(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:20,790 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 234d84c9a604cb1d5f1618efa32bcb73: 2023-02-15 06:54:20,790 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73., storeName=234d84c9a604cb1d5f1618efa32bcb73/cf1, priority=9, startTime=1676444060730; duration=0sec 2023-02-15 06:54:20,790 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:20,798 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/.tmp/cf2/2601b4bc9c6f4a75a6571281bf50aee3 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/2601b4bc9c6f4a75a6571281bf50aee3 2023-02-15 06:54:20,803 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 234d84c9a604cb1d5f1618efa32bcb73/cf2 of 234d84c9a604cb1d5f1618efa32bcb73 into 2601b4bc9c6f4a75a6571281bf50aee3(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:20,803 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 234d84c9a604cb1d5f1618efa32bcb73: 2023-02-15 06:54:20,803 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73., storeName=234d84c9a604cb1d5f1618efa32bcb73/cf2, priority=9, startTime=1676444060730; duration=0sec 2023-02-15 06:54:20,803 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:20,808 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2-top 2023-02-15 06:54:20,811 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:20,811 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1-top 2023-02-15 06:54:20,821 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71-top 2023-02-15 06:54:20,824 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83-top 2023-02-15 06:54:20,828 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ce24cc23e9c422c9f2f96980442e4a9 2023-02-15 06:54:20,828 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5ce24cc23e9c422c9f2f96980442e4a9 2023-02-15 06:54:20,831 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b6c1c2ca40a49cc8b801e45ce4dea2b 2023-02-15 06:54:20,831 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-6b6c1c2ca40a49cc8b801e45ce4dea2b 2023-02-15 06:54:20,834 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bece57e3ef26499cbd533c0591ba291d 2023-02-15 06:54:20,834 DEBUG [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-bece57e3ef26499cbd533c0591ba291d 2023-02-15 06:54:20,835 INFO [StoreOpener-e51419864fae0902e5f91adf742ee434-1] regionserver.HStore(310): Store=e51419864fae0902e5f91adf742ee434/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:20,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:20,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:20,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:54:20,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:20,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e51419864fae0902e5f91adf742ee434; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66103949, jitterRate=-0.014974400401115417}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:54:20,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e51419864fae0902e5f91adf742ee434: 2023-02-15 06:54:20,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434., pid=77, masterSystemTime=1676444060635 2023-02-15 06:54:20,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:20,841 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:54:20,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:54:20,841 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:54:20,841 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:20,841 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1912): e51419864fae0902e5f91adf742ee434/cf1 is initiating minor compaction (all files) 2023-02-15 06:54:20,841 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:20,841 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1912): e51419864fae0902e5f91adf742ee434/cf2 is initiating minor compaction (all files) 2023-02-15 06:54:20,841 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e51419864fae0902e5f91adf742ee434/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:20,841 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e51419864fae0902e5f91adf742ee434/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:20,841 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-d9921529217e4617b85a44197272bd7e, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-0f5666bd44054b28b040f250412c0736, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5cad67575de04fd3b9952f726b573ffc] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/.tmp, totalSize=34.8 K 2023-02-15 06:54:20,842 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-bece57e3ef26499cbd533c0591ba291d, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5ce24cc23e9c422c9f2f96980442e4a9, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-6b6c1c2ca40a49cc8b801e45ce4dea2b] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/.tmp, totalSize=34.8 K 2023-02-15 06:54:20,842 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676444058115 2023-02-15 06:54:20,842 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676444058115 2023-02-15 06:54:20,842 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676444058295 2023-02-15 06:54:20,843 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676444058295 2023-02-15 06:54:20,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:20,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:20,843 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676444058506 2023-02-15 06:54:20,843 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e51419864fae0902e5f91adf742ee434, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:20,843 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676444058506 2023-02-15 06:54:20,843 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676444059258 2023-02-15 06:54:20,843 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676444060843"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444060843"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444060843"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444060843"}]},"ts":"1676444060843"} 2023-02-15 06:54:20,844 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676444059258 2023-02-15 06:54:20,844 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-d9921529217e4617b85a44197272bd7e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-15 06:54:20,844 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-bece57e3ef26499cbd533c0591ba291d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-15 06:54:20,844 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-0f5666bd44054b28b040f250412c0736, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-15 06:54:20,845 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5ce24cc23e9c422c9f2f96980442e4a9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-15 06:54:20,845 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5cad67575de04fd3b9952f726b573ffc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-15 06:54:20,845 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-6b6c1c2ca40a49cc8b801e45ce4dea2b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-15 06:54:20,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-15 06:54:20,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure e51419864fae0902e5f91adf742ee434, server=jenkins-hbase4.apache.org,40475,1676444032771 in 361 msec 2023-02-15 06:54:20,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-15 06:54:20,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e51419864fae0902e5f91adf742ee434, ASSIGN in 521 msec 2023-02-15 06:54:20,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 in 1.0360 sec 2023-02-15 06:54:20,862 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] throttle.PressureAwareThroughputController(145): e51419864fae0902e5f91adf742ee434#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:20,864 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] throttle.PressureAwareThroughputController(145): e51419864fae0902e5f91adf742ee434#cf2#compaction#75 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:20,886 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for df0a7c0e1dbd4059b934ad325142f3ea: NONE, but ROW specified in column family configuration 2023-02-15 06:54:20,886 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/.tmp/cf1/df0a7c0e1dbd4059b934ad325142f3ea as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/df0a7c0e1dbd4059b934ad325142f3ea 2023-02-15 06:54:20,891 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for df0a7c0e1dbd4059b934ad325142f3ea: NONE, but ROW specified in column family configuration 2023-02-15 06:54:20,892 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e51419864fae0902e5f91adf742ee434/cf1 of e51419864fae0902e5f91adf742ee434 into df0a7c0e1dbd4059b934ad325142f3ea(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:20,892 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e51419864fae0902e5f91adf742ee434: 2023-02-15 06:54:20,892 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434., storeName=e51419864fae0902e5f91adf742ee434/cf1, priority=9, startTime=1676444060840; duration=0sec 2023-02-15 06:54:20,892 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:20,899 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for b5017243cc6c4057b5a1e042ec0a2d91: NONE, but ROW specified in column family configuration 2023-02-15 06:54:20,900 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/.tmp/cf2/b5017243cc6c4057b5a1e042ec0a2d91 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/b5017243cc6c4057b5a1e042ec0a2d91 2023-02-15 06:54:20,904 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for b5017243cc6c4057b5a1e042ec0a2d91: NONE, but ROW specified in column family configuration 2023-02-15 06:54:20,905 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e51419864fae0902e5f91adf742ee434/cf2 of e51419864fae0902e5f91adf742ee434 into b5017243cc6c4057b5a1e042ec0a2d91(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:20,905 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e51419864fae0902e5f91adf742ee434: 2023-02-15 06:54:20,905 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434., storeName=e51419864fae0902e5f91adf742ee434/cf2, priority=9, startTime=1676444060841; duration=0sec 2023-02-15 06:54:20,905 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:21,076 DEBUG [Listener at localhost/37669] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-15 06:54:21,079 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59824, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-15 06:54:21,079 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40475] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:21,080 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40475] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-15 06:54:21,080 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40475] regionserver.HStore(1912): 234d84c9a604cb1d5f1618efa32bcb73/cf1 is initiating major compaction (all files) 2023-02-15 06:54:21,081 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40475] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@22c0ea88; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:21,081 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 234d84c9a604cb1d5f1618efa32bcb73/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:21,081 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40475] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-15 06:54:21,081 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/17324af8340241d2b493da2644597f77] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/.tmp, totalSize=6.8 K 2023-02-15 06:54:21,081 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40475] regionserver.HStore(1912): 234d84c9a604cb1d5f1618efa32bcb73/cf2 is initiating major compaction (all files) 2023-02-15 06:54:21,081 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40475] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@e7c575d; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:54:21,081 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 234d84c9a604cb1d5f1618efa32bcb73/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:21,081 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/2601b4bc9c6f4a75a6571281bf50aee3] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/.tmp, totalSize=6.8 K 2023-02-15 06:54:21,081 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 17324af8340241d2b493da2644597f77, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676444058115 2023-02-15 06:54:21,082 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 2601b4bc9c6f4a75a6571281bf50aee3, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676444058115 2023-02-15 06:54:21,084 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40475] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:21,084 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40475] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-15 06:54:21,084 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40475] regionserver.HStore(1912): e51419864fae0902e5f91adf742ee434/cf1 is initiating major compaction (all files) 2023-02-15 06:54:21,084 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40475] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@36b64388; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:54:21,084 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40475] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-15 06:54:21,084 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40475] regionserver.HStore(1912): e51419864fae0902e5f91adf742ee434/cf2 is initiating major compaction (all files) 2023-02-15 06:54:21,084 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40475] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3a8a8896; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-15 06:54:21,085 INFO [Listener at localhost/37669] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-15 06:54:21,086 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] throttle.PressureAwareThroughputController(145): 234d84c9a604cb1d5f1618efa32bcb73#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:21,088 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] throttle.PressureAwareThroughputController(145): 234d84c9a604cb1d5f1618efa32bcb73#cf2#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:21,103 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40475] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. is not mergeable because it has references 2023-02-15 06:54:21,103 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40475] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. is not mergeable because it has references 2023-02-15 06:54:21,104 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/.tmp/cf1/54e2fb11e16843908848bd5cf5e4421b as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/54e2fb11e16843908848bd5cf5e4421b 2023-02-15 06:54:21,109 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 234d84c9a604cb1d5f1618efa32bcb73/cf1 of 234d84c9a604cb1d5f1618efa32bcb73 into 54e2fb11e16843908848bd5cf5e4421b(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:21,109 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 234d84c9a604cb1d5f1618efa32bcb73: 2023-02-15 06:54:21,111 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676444061080; duration=0sec 2023-02-15 06:54:21,111 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-15 06:54:21,111 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e51419864fae0902e5f91adf742ee434/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:21,111 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/b5017243cc6c4057b5a1e042ec0a2d91] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/.tmp, totalSize=5.0 K 2023-02-15 06:54:21,111 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting b5017243cc6c4057b5a1e042ec0a2d91, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-15 06:54:21,114 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] throttle.PressureAwareThroughputController(145): e51419864fae0902e5f91adf742ee434#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:21,124 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/.tmp/cf2/4538b6304ed24328b41afd5d24733747 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/4538b6304ed24328b41afd5d24733747 2023-02-15 06:54:21,128 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 38e90601e0e94f1e9faa958192aaf170: NONE, but ROW specified in column family configuration 2023-02-15 06:54:21,129 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 234d84c9a604cb1d5f1618efa32bcb73/cf2 of 234d84c9a604cb1d5f1618efa32bcb73 into 4538b6304ed24328b41afd5d24733747(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:21,129 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/.tmp/cf2/38e90601e0e94f1e9faa958192aaf170 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/38e90601e0e94f1e9faa958192aaf170 2023-02-15 06:54:21,129 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 234d84c9a604cb1d5f1618efa32bcb73: 2023-02-15 06:54:21,129 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676444061081; duration=0sec 2023-02-15 06:54:21,129 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:54:21,130 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e51419864fae0902e5f91adf742ee434/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:21,130 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/df0a7c0e1dbd4059b934ad325142f3ea] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/.tmp, totalSize=5.0 K 2023-02-15 06:54:21,130 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting df0a7c0e1dbd4059b934ad325142f3ea, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-15 06:54:21,133 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] throttle.PressureAwareThroughputController(145): e51419864fae0902e5f91adf742ee434#cf1#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:54:21,133 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 38e90601e0e94f1e9faa958192aaf170: NONE, but ROW specified in column family configuration 2023-02-15 06:54:21,134 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e51419864fae0902e5f91adf742ee434/cf2 of e51419864fae0902e5f91adf742ee434 into 38e90601e0e94f1e9faa958192aaf170(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:21,134 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e51419864fae0902e5f91adf742ee434: 2023-02-15 06:54:21,135 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676444061084; duration=0sec 2023-02-15 06:54:21,135 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:21,146 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for de706eef68314e5a80463c45d0e2acc1: NONE, but ROW specified in column family configuration 2023-02-15 06:54:21,147 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/.tmp/cf1/de706eef68314e5a80463c45d0e2acc1 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/de706eef68314e5a80463c45d0e2acc1 2023-02-15 06:54:21,152 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for de706eef68314e5a80463c45d0e2acc1: NONE, but ROW specified in column family configuration 2023-02-15 06:54:21,153 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e51419864fae0902e5f91adf742ee434/cf1 of e51419864fae0902e5f91adf742ee434 into de706eef68314e5a80463c45d0e2acc1(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:54:21,153 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e51419864fae0902e5f91adf742ee434: 2023-02-15 06:54:21,153 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676444061084; duration=0sec 2023-02-15 06:54:21,154 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:54:21,212 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40475] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. is not mergeable because it has references 2023-02-15 06:54:21,213 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40475] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. is not mergeable because it has references 2023-02-15 06:54:21,218 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-15 06:54:21,220 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:21,222 INFO [Listener at localhost/37669] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:21,222 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:21,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:21,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-15 06:54:21,225 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444061225"}]},"ts":"1676444061225"} 2023-02-15 06:54:21,226 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-15 06:54:21,228 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-15 06:54:21,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=234d84c9a604cb1d5f1618efa32bcb73, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e51419864fae0902e5f91adf742ee434, UNASSIGN}] 2023-02-15 06:54:21,229 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e51419864fae0902e5f91adf742ee434, UNASSIGN 2023-02-15 06:54:21,230 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=e51419864fae0902e5f91adf742ee434, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:21,230 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676444061230"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444061230"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444061230"}]},"ts":"1676444061230"} 2023-02-15 06:54:21,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure e51419864fae0902e5f91adf742ee434, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:21,233 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=234d84c9a604cb1d5f1618efa32bcb73, UNASSIGN 2023-02-15 06:54:21,234 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=234d84c9a604cb1d5f1618efa32bcb73, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:54:21,234 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676444061234"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444061234"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444061234"}]},"ts":"1676444061234"} 2023-02-15 06:54:21,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 234d84c9a604cb1d5f1618efa32bcb73, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:54:21,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-15 06:54:21,383 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:21,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e51419864fae0902e5f91adf742ee434, disabling compactions & flushes 2023-02-15 06:54:21,383 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:21,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:21,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. after waiting 0 ms 2023-02-15 06:54:21,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:21,384 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-d9921529217e4617b85a44197272bd7e, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-0f5666bd44054b28b040f250412c0736, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/df0a7c0e1dbd4059b934ad325142f3ea, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5cad67575de04fd3b9952f726b573ffc] to archive 2023-02-15 06:54:21,385 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:21,387 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,388 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,390 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,391 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,392 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-d9921529217e4617b85a44197272bd7e to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-d9921529217e4617b85a44197272bd7e 2023-02-15 06:54:21,393 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-0f5666bd44054b28b040f250412c0736 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-0f5666bd44054b28b040f250412c0736 2023-02-15 06:54:21,394 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/df0a7c0e1dbd4059b934ad325142f3ea to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/df0a7c0e1dbd4059b934ad325142f3ea 2023-02-15 06:54:21,395 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5cad67575de04fd3b9952f726b573ffc to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5cad67575de04fd3b9952f726b573ffc 2023-02-15 06:54:21,397 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-bece57e3ef26499cbd533c0591ba291d, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5ce24cc23e9c422c9f2f96980442e4a9, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/b5017243cc6c4057b5a1e042ec0a2d91, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-6b6c1c2ca40a49cc8b801e45ce4dea2b] to archive 2023-02-15 06:54:21,398 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:21,399 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,400 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,401 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,402 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,403 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-bece57e3ef26499cbd533c0591ba291d to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-bece57e3ef26499cbd533c0591ba291d 2023-02-15 06:54:21,404 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5ce24cc23e9c422c9f2f96980442e4a9 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5ce24cc23e9c422c9f2f96980442e4a9 2023-02-15 06:54:21,405 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/b5017243cc6c4057b5a1e042ec0a2d91 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/b5017243cc6c4057b5a1e042ec0a2d91 2023-02-15 06:54:21,406 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-6b6c1c2ca40a49cc8b801e45ce4dea2b to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-6b6c1c2ca40a49cc8b801e45ce4dea2b 2023-02-15 06:54:21,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-15 06:54:21,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:21,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434. 2023-02-15 06:54:21,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e51419864fae0902e5f91adf742ee434: 2023-02-15 06:54:21,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:21,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:21,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 234d84c9a604cb1d5f1618efa32bcb73, disabling compactions & flushes 2023-02-15 06:54:21,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:21,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:21,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. after waiting 0 ms 2023-02-15 06:54:21,413 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=e51419864fae0902e5f91adf742ee434, regionState=CLOSED 2023-02-15 06:54:21,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:21,413 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676444061413"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444061413"}]},"ts":"1676444061413"} 2023-02-15 06:54:21,414 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5099df7157024afaa6d2084ad018b5ce, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-3164a2aa76b44799aebc4ab7b85cefaa, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-12535f15786c43b88367bcbc1f01d0da, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/17324af8340241d2b493da2644597f77, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102-bottom] to archive 2023-02-15 06:54:21,415 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:21,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-15 06:54:21,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure e51419864fae0902e5f91adf742ee434, server=jenkins-hbase4.apache.org,40475,1676444032771 in 184 msec 2023-02-15 06:54:21,417 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/3677f20b900f4f4aa7a6e200b103aa9d.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e51419864fae0902e5f91adf742ee434, UNASSIGN in 189 msec 2023-02-15 06:54:21,418 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/c595e942710147dab770ee17b56584be.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,420 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/9c9b9f0c05974a31859444e3450d35ae.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,421 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5099df7157024afaa6d2084ad018b5ce to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-5099df7157024afaa6d2084ad018b5ce 2023-02-15 06:54:21,422 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-3164a2aa76b44799aebc4ab7b85cefaa to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-3164a2aa76b44799aebc4ab7b85cefaa 2023-02-15 06:54:21,423 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-12535f15786c43b88367bcbc1f01d0da to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-12535f15786c43b88367bcbc1f01d0da 2023-02-15 06:54:21,424 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/17324af8340241d2b493da2644597f77 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/17324af8340241d2b493da2644597f77 2023-02-15 06:54:21,425 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/2196511bb07747c585a68b86f2d1b102.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,427 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-441c7b821ac741aa83d0c66b64613bc5, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-8ec47342b8a34ce6ba11286bdf7eb268, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-4781241327b641f488ad79fc8157b5f3, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/2601b4bc9c6f4a75a6571281bf50aee3, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1-bottom] to archive 2023-02-15 06:54:21,427 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:54:21,429 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/18df5422413049f5be21d718b03e8cc2.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,430 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/c0212b1c94184df2a46581dbd45b6c83.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,431 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/792fe3b1080e4a34a94b4abb457adf71.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,432 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-441c7b821ac741aa83d0c66b64613bc5 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-441c7b821ac741aa83d0c66b64613bc5 2023-02-15 06:54:21,434 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-8ec47342b8a34ce6ba11286bdf7eb268 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-8ec47342b8a34ce6ba11286bdf7eb268 2023-02-15 06:54:21,435 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-4781241327b641f488ad79fc8157b5f3 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/testSplitTableRegionDeletedRowsDaughter=a93cfa0b8140cbc18c2cd8f565cfd003-4781241327b641f488ad79fc8157b5f3 2023-02-15 06:54:21,436 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/2601b4bc9c6f4a75a6571281bf50aee3 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/2601b4bc9c6f4a75a6571281bf50aee3 2023-02-15 06:54:21,437 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/727bdd9c8dbe4c31a84af2fa728250b1.a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-15 06:54:21,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:21,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73. 2023-02-15 06:54:21,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 234d84c9a604cb1d5f1618efa32bcb73: 2023-02-15 06:54:21,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:21,443 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=234d84c9a604cb1d5f1618efa32bcb73, regionState=CLOSED 2023-02-15 06:54:21,443 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676444061443"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444061443"}]},"ts":"1676444061443"} 2023-02-15 06:54:21,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-15 06:54:21,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 234d84c9a604cb1d5f1618efa32bcb73, server=jenkins-hbase4.apache.org,40475,1676444032771 in 209 msec 2023-02-15 06:54:21,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-15 06:54:21,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=234d84c9a604cb1d5f1618efa32bcb73, UNASSIGN in 218 msec 2023-02-15 06:54:21,448 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444061448"}]},"ts":"1676444061448"} 2023-02-15 06:54:21,449 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-15 06:54:21,451 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-15 06:54:21,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 230 msec 2023-02-15 06:54:21,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-15 06:54:21,528 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-15 06:54:21,528 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:21,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:21,531 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:21,531 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:21,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-15 06:54:21,535 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,535 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:21,535 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:21,537 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/recovered.edits] 2023-02-15 06:54:21,537 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/recovered.edits] 2023-02-15 06:54:21,537 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/recovered.edits] 2023-02-15 06:54:21,540 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/54e2fb11e16843908848bd5cf5e4421b to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf1/54e2fb11e16843908848bd5cf5e4421b 2023-02-15 06:54:21,540 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/de706eef68314e5a80463c45d0e2acc1 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf1/de706eef68314e5a80463c45d0e2acc1 2023-02-15 06:54:21,540 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/0f5666bd44054b28b040f250412c0736 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/0f5666bd44054b28b040f250412c0736 2023-02-15 06:54:21,542 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/12535f15786c43b88367bcbc1f01d0da to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/12535f15786c43b88367bcbc1f01d0da 2023-02-15 06:54:21,543 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/38e90601e0e94f1e9faa958192aaf170 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/cf2/38e90601e0e94f1e9faa958192aaf170 2023-02-15 06:54:21,543 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/4538b6304ed24328b41afd5d24733747 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/cf2/4538b6304ed24328b41afd5d24733747 2023-02-15 06:54:21,543 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/2196511bb07747c585a68b86f2d1b102 2023-02-15 06:54:21,545 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3164a2aa76b44799aebc4ab7b85cefaa to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3164a2aa76b44799aebc4ab7b85cefaa 2023-02-15 06:54:21,546 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/3677f20b900f4f4aa7a6e200b103aa9d 2023-02-15 06:54:21,546 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/recovered.edits/113.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73/recovered.edits/113.seqid 2023-02-15 06:54:21,547 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/recovered.edits/113.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434/recovered.edits/113.seqid 2023-02-15 06:54:21,547 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/234d84c9a604cb1d5f1618efa32bcb73 2023-02-15 06:54:21,547 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e51419864fae0902e5f91adf742ee434 2023-02-15 06:54:21,548 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5099df7157024afaa6d2084ad018b5ce to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5099df7157024afaa6d2084ad018b5ce 2023-02-15 06:54:21,549 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5cad67575de04fd3b9952f726b573ffc to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/5cad67575de04fd3b9952f726b573ffc 2023-02-15 06:54:21,550 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/9c9b9f0c05974a31859444e3450d35ae 2023-02-15 06:54:21,551 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/c595e942710147dab770ee17b56584be 2023-02-15 06:54:21,553 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/d9921529217e4617b85a44197272bd7e to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf1/d9921529217e4617b85a44197272bd7e 2023-02-15 06:54:21,555 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/18df5422413049f5be21d718b03e8cc2 2023-02-15 06:54:21,556 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/441c7b821ac741aa83d0c66b64613bc5 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/441c7b821ac741aa83d0c66b64613bc5 2023-02-15 06:54:21,558 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/4781241327b641f488ad79fc8157b5f3 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/4781241327b641f488ad79fc8157b5f3 2023-02-15 06:54:21,559 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/5ce24cc23e9c422c9f2f96980442e4a9 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/5ce24cc23e9c422c9f2f96980442e4a9 2023-02-15 06:54:21,560 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/6b6c1c2ca40a49cc8b801e45ce4dea2b to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/6b6c1c2ca40a49cc8b801e45ce4dea2b 2023-02-15 06:54:21,561 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/727bdd9c8dbe4c31a84af2fa728250b1 2023-02-15 06:54:21,562 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/792fe3b1080e4a34a94b4abb457adf71 2023-02-15 06:54:21,563 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/8ec47342b8a34ce6ba11286bdf7eb268 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/8ec47342b8a34ce6ba11286bdf7eb268 2023-02-15 06:54:21,564 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/bece57e3ef26499cbd533c0591ba291d to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/bece57e3ef26499cbd533c0591ba291d 2023-02-15 06:54:21,565 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/cf2/c0212b1c94184df2a46581dbd45b6c83 2023-02-15 06:54:21,567 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/recovered.edits/106.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003/recovered.edits/106.seqid 2023-02-15 06:54:21,568 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/a93cfa0b8140cbc18c2cd8f565cfd003 2023-02-15 06:54:21,568 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-15 06:54:21,570 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:21,572 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-15 06:54:21,574 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-15 06:54:21,575 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:21,575 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-15 06:54:21,575 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444061575"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:21,575 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444061575"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:21,575 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444061575"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:21,577 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-15 06:54:21,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a93cfa0b8140cbc18c2cd8f565cfd003, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676444057610.a93cfa0b8140cbc18c2cd8f565cfd003.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 234d84c9a604cb1d5f1618efa32bcb73, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676444059814.234d84c9a604cb1d5f1618efa32bcb73.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => e51419864fae0902e5f91adf742ee434, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676444059814.e51419864fae0902e5f91adf742ee434.', STARTKEY => '60', ENDKEY => ''}] 2023-02-15 06:54:21,577 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-15 06:54:21,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676444061577"}]},"ts":"9223372036854775807"} 2023-02-15 06:54:21,578 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-15 06:54:21,580 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:54:21,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 52 msec 2023-02-15 06:54:21,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-15 06:54:21,633 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-15 06:54:21,647 INFO [Listener at localhost/37669] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=411 (was 406) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:44610 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:36676 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:41842 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:44872 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:37190 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:40475-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:44526 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:44634 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:42166 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:41928 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:53456 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:45016 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:42130 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:40084 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:44878 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:49230 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_954404014_15 at /127.0.0.1:53672 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=842 (was 840) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=252 (was 249) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3294 (was 3310) 2023-02-15 06:54:21,662 INFO [Listener at localhost/37669] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=411, OpenFileDescriptor=842, MaxFileDescriptor=60000, SystemLoadAverage=252, ProcessCount=168, AvailableMemoryMB=3293 2023-02-15 06:54:21,662 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:54:21,665 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-15 06:54:21,665 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-15 06:54:21,666 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:54:21,667 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:54:21,668 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:54:21,669 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320 2023-02-15 06:54:21,670 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320 empty. 2023-02-15 06:54:21,670 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320 2023-02-15 06:54:21,670 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-15 06:54:21,684 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-15 06:54:21,685 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => c6d354efa00d71a16486322a96778320, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:54:21,694 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:21,694 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing c6d354efa00d71a16486322a96778320, disabling compactions & flushes 2023-02-15 06:54:21,694 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:21,694 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:21,694 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. after waiting 0 ms 2023-02-15 06:54:21,694 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:21,694 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:21,694 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for c6d354efa00d71a16486322a96778320: 2023-02-15 06:54:21,696 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:54:21,697 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444061697"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444061697"}]},"ts":"1676444061697"} 2023-02-15 06:54:21,698 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:54:21,699 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:54:21,699 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444061699"}]},"ts":"1676444061699"} 2023-02-15 06:54:21,700 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-15 06:54:21,703 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:54:21,704 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:54:21,704 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:54:21,704 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:54:21,704 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:54:21,704 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, ASSIGN}] 2023-02-15 06:54:21,705 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, ASSIGN 2023-02-15 06:54:21,706 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:54:21,856 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:54:21,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=c6d354efa00d71a16486322a96778320, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:21,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444061859"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444061859"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444061859"}]},"ts":"1676444061859"} 2023-02-15 06:54:21,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure c6d354efa00d71a16486322a96778320, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:54:22,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:22,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c6d354efa00d71a16486322a96778320, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:54:22,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:22,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:22,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,020 INFO [StoreOpener-c6d354efa00d71a16486322a96778320-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,021 DEBUG [StoreOpener-c6d354efa00d71a16486322a96778320-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/cf1 2023-02-15 06:54:22,021 DEBUG [StoreOpener-c6d354efa00d71a16486322a96778320-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/cf1 2023-02-15 06:54:22,022 INFO [StoreOpener-c6d354efa00d71a16486322a96778320-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c6d354efa00d71a16486322a96778320 columnFamilyName cf1 2023-02-15 06:54:22,022 INFO [StoreOpener-c6d354efa00d71a16486322a96778320-1] regionserver.HStore(310): Store=c6d354efa00d71a16486322a96778320/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:22,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:54:22,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c6d354efa00d71a16486322a96778320; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62198530, jitterRate=-0.07316967844963074}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-15 06:54:22,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c6d354efa00d71a16486322a96778320: 2023-02-15 06:54:22,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320., pid=87, masterSystemTime=1676444062012 2023-02-15 06:54:22,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:22,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:22,031 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=c6d354efa00d71a16486322a96778320, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:22,031 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444062031"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444062031"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444062031"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444062031"}]},"ts":"1676444062031"} 2023-02-15 06:54:22,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-15 06:54:22,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure c6d354efa00d71a16486322a96778320, server=jenkins-hbase4.apache.org,41759,1676444032818 in 172 msec 2023-02-15 06:54:22,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-15 06:54:22,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, ASSIGN in 330 msec 2023-02-15 06:54:22,036 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:54:22,036 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444062036"}]},"ts":"1676444062036"} 2023-02-15 06:54:22,038 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-15 06:54:22,044 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:54:22,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 379 msec 2023-02-15 06:54:22,176 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-15 06:54:22,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:54:22,194 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-15 06:54:22,195 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-15 06:54:22,195 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-15 06:54:22,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-15 06:54:22,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, REOPEN/MOVE}] 2023-02-15 06:54:22,211 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, REOPEN/MOVE 2023-02-15 06:54:22,211 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=c6d354efa00d71a16486322a96778320, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:22,211 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444062211"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444062211"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444062211"}]},"ts":"1676444062211"} 2023-02-15 06:54:22,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure c6d354efa00d71a16486322a96778320, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:54:22,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c6d354efa00d71a16486322a96778320, disabling compactions & flushes 2023-02-15 06:54:22,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:22,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:22,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. after waiting 0 ms 2023-02-15 06:54:22,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:22,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-15 06:54:22,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:54:22,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:22,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c6d354efa00d71a16486322a96778320: 2023-02-15 06:54:22,370 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: c6d354efa00d71a16486322a96778320 to self. 2023-02-15 06:54:22,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,372 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=c6d354efa00d71a16486322a96778320, regionState=CLOSED 2023-02-15 06:54:22,372 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444062372"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444062372"}]},"ts":"1676444062372"} 2023-02-15 06:54:22,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-15 06:54:22,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure c6d354efa00d71a16486322a96778320, server=jenkins-hbase4.apache.org,41759,1676444032818 in 161 msec 2023-02-15 06:54:22,375 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=true 2023-02-15 06:54:22,525 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:54:22,526 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=c6d354efa00d71a16486322a96778320, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:22,526 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444062526"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444062526"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444062526"}]},"ts":"1676444062526"} 2023-02-15 06:54:22,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure c6d354efa00d71a16486322a96778320, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:54:22,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:22,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c6d354efa00d71a16486322a96778320, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:54:22,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:22,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:22,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,684 INFO [StoreOpener-c6d354efa00d71a16486322a96778320-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,685 DEBUG [StoreOpener-c6d354efa00d71a16486322a96778320-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/cf1 2023-02-15 06:54:22,685 DEBUG [StoreOpener-c6d354efa00d71a16486322a96778320-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/cf1 2023-02-15 06:54:22,685 INFO [StoreOpener-c6d354efa00d71a16486322a96778320-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c6d354efa00d71a16486322a96778320 columnFamilyName cf1 2023-02-15 06:54:22,686 INFO [StoreOpener-c6d354efa00d71a16486322a96778320-1] regionserver.HStore(310): Store=c6d354efa00d71a16486322a96778320/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:54:22,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c6d354efa00d71a16486322a96778320 2023-02-15 06:54:22,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c6d354efa00d71a16486322a96778320; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71504812, jitterRate=0.06550472974777222}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-15 06:54:22,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c6d354efa00d71a16486322a96778320: 2023-02-15 06:54:22,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320., pid=92, masterSystemTime=1676444062680 2023-02-15 06:54:22,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:22,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:54:22,692 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=c6d354efa00d71a16486322a96778320, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:54:22,692 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444062692"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444062692"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444062692"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444062692"}]},"ts":"1676444062692"} 2023-02-15 06:54:22,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-15 06:54:22,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure c6d354efa00d71a16486322a96778320, server=jenkins-hbase4.apache.org,41759,1676444032818 in 166 msec 2023-02-15 06:54:22,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-15 06:54:22,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, REOPEN/MOVE in 486 msec 2023-02-15 06:54:22,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-15 06:54:22,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 501 msec 2023-02-15 06:54:22,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a, ASSIGN}] 2023-02-15 06:54:22,702 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a, ASSIGN 2023-02-15 06:54:22,703 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-15 06:54:22,853 DEBUG [jenkins-hbase4:46669] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:54:22,855 DEBUG [jenkins-hbase4:46669] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:54:22,855 DEBUG [jenkins-hbase4:46669] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:54:22,855 DEBUG [jenkins-hbase4:46669] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:54:22,855 DEBUG [jenkins-hbase4:46669] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-15 06:54:22,857 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=dd4f5e76327a2bdd0e2eb11acd24927a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:54:22,857 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444062857"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676444062857"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676444062857"}]},"ts":"1676444062857"} 2023-02-15 06:54:22,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure dd4f5e76327a2bdd0e2eb11acd24927a, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:54:23,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a. 2023-02-15 06:54:23,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => dd4f5e76327a2bdd0e2eb11acd24927a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-15 06:54:23,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:54:23,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica dd4f5e76327a2bdd0e2eb11acd24927a 2023-02-15 06:54:23,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:54:23,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for dd4f5e76327a2bdd0e2eb11acd24927a 2023-02-15 06:54:23,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for dd4f5e76327a2bdd0e2eb11acd24927a 2023-02-15 06:54:23,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:23,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:23,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:23,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:23,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:23,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:23,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:23,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:23,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:23,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:24,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:24,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:24,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:24,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:24,327 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-15 06:54:24,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:24,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:24,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:24,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:24,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:24,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:25,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:25,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:25,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:25,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:25,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:25,514 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-15 06:54:25,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:25,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:25,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:25,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:25,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:26,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:26,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:26,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:26,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:26,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:26,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:26,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:26,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:26,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:26,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:27,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:27,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:27,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:27,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:27,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:27,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:27,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:27,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:27,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:27,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:28,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:28,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:28,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:28,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:28,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:28,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:28,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:28,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:28,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:28,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:29,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:29,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:29,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:29,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:29,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:29,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:29,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:29,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:29,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:29,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:30,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:30,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:30,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:30,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:30,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:30,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:30,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:30,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:30,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:30,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:31,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:31,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:31,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:31,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:31,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:31,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:31,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:31,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:31,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:31,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:32,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:32,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:32,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:32,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:32,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:32,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:32,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:32,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:32,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:32,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:33,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:33,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:33,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:33,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:33,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:33,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:33,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:33,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:33,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:33,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:34,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:34,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:34,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:34,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:34,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:34,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:34,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:34,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:34,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:34,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:35,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:35,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:35,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:35,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:35,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:35,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:35,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:35,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:35,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:35,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:36,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:36,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:36,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:36,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:36,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:36,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:36,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:36,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:36,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:36,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:37,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:37,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:37,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:37,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:37,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:37,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:37,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:37,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:37,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:37,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:38,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:38,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:38,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:38,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:38,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:38,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:38,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:38,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:38,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:38,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:39,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:39,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:39,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:39,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:39,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:39,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:39,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:39,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:39,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:39,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:40,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:40,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:40,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:40,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:40,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:40,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:40,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:40,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:40,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:40,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:41,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:41,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:41,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:41,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:41,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:41,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:41,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:41,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:41,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:41,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:42,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:42,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:42,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:42,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:42,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:42,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:42,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:42,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:42,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:42,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:43,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:43,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:43,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:43,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:43,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:43,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:43,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:43,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:43,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:43,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:44,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:44,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:44,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:44,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:44,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:44,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:44,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:44,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:44,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:44,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:45,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:45,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:45,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:45,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:45,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:45,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:45,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:45,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:45,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:45,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:46,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:46,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:46,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:46,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:46,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:46,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:46,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:46,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:46,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:46,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:47,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:47,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:47,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:47,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:47,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:47,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:47,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:47,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:47,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:47,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:48,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:48,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:48,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:48,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:48,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:48,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:48,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:48,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:48,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:48,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:49,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:49,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:49,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:49,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:49,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:49,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:49,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:49,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:49,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:49,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:50,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:50,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:50,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:50,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:50,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:50,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:50,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:50,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:50,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:50,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:51,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:51,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:51,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:51,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:51,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:51,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:51,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:51,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:51,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:51,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:52,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:52,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:52,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:52,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:52,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:52,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:52,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:52,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:52,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:52,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:53,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:53,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:53,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:53,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:53,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:53,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:53,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:53,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:53,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:53,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:54,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:54,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:54,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:54,372 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-15 06:54:54,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:54,430 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-15 06:54:54,436 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-15 06:54:54,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:54,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:54,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:54,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:54,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:54,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:55,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:55,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:55,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:55,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:55,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:55,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:55,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:55,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:55,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:55,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:56,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:56,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:56,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:56,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:56,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:56,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:56,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:56,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:56,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:56,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:57,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:57,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:57,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:57,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:57,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:57,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:57,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:57,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:57,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:57,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:58,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:58,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:58,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:58,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:58,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:58,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:58,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:58,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:58,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:58,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:59,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:59,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:59,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:59,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:59,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:59,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:59,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:59,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:59,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:54:59,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:00,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:00,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:00,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:00,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:00,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:00,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:00,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:00,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:00,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:00,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:01,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:01,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:01,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:01,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:01,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:01,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:01,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:01,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:01,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:01,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:02,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:02,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:02,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:02,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:02,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:02,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:02,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:02,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:02,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:02,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:03,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:03,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:03,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:03,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:03,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:03,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:03,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:03,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:03,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:04,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:04,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:04,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:04,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:04,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:04,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:04,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:04,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:04,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:04,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:05,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:05,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:05,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:05,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:05,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:05,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:05,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:05,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:05,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:05,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:06,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:06,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:06,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:06,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:06,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:06,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:06,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:06,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:06,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:06,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:07,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:07,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:07,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:07,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:07,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:07,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:07,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:07,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:07,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:07,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:08,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:08,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:08,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:08,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:08,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:08,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:08,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:08,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:08,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:08,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:09,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:09,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:09,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:09,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:09,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:09,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:09,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:09,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:09,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:09,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:10,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:10,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:10,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:10,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:10,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:10,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:10,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:10,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:10,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:10,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:11,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:11,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:11,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:11,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:11,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:11,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:11,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:11,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:11,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:11,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:12,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:12,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:12,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:12,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:12,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:12,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:12,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:12,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:12,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:12,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:13,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:13,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:13,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:13,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:13,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:13,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:13,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:13,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:13,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:13,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:14,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:14,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:14,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:14,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:14,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:14,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:14,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:14,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:14,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:14,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:15,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:15,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:15,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:15,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:15,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:15,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:15,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:15,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:15,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:15,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:16,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:16,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:16,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:16,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:16,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:16,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:16,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:16,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:16,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:16,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:17,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:17,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:17,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:17,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:17,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:17,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:17,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:17,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:17,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:17,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:18,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:18,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:18,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:18,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:18,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:18,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:18,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:18,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:18,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:18,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:19,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:19,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:19,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:19,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:19,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:19,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:19,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:19,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:19,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:19,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:20,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:20,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:20,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:20,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:20,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:20,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:20,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:20,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:20,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:20,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:21,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:21,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:21,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:21,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:21,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:21,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:21,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:21,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:21,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:21,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:22,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:22,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:22,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:22,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-15 06:55:22,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:22,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-15 06:55:22,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:22,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:22,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-15 06:55:22,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:22,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:22,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:22,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-15 06:55:22,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:23,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:23,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:23,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:23,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:23,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-15 06:55:23,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:23,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:23,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:23,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:23,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:23,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:24,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:24,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:24,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:24,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:24,397 DEBUG [Listener at localhost/37669] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:24,398 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:24,399 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:24,399 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:24,399 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:24,401 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting c6d354efa00d71a16486322a96778320, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:24,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 exec-time=5 msec 2023-02-15 06:55:24,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:24,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:24,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-15 06:55:24,744 INFO [StoreOpener-dd4f5e76327a2bdd0e2eb11acd24927a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region dd4f5e76327a2bdd0e2eb11acd24927a 2023-02-15 06:55:24,747 DEBUG [StoreOpener-dd4f5e76327a2bdd0e2eb11acd24927a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/cf1 2023-02-15 06:55:24,747 DEBUG [StoreOpener-dd4f5e76327a2bdd0e2eb11acd24927a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/cf1 2023-02-15 06:55:24,748 INFO [StoreOpener-dd4f5e76327a2bdd0e2eb11acd24927a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd4f5e76327a2bdd0e2eb11acd24927a columnFamilyName cf1 2023-02-15 06:55:24,749 INFO [StoreOpener-dd4f5e76327a2bdd0e2eb11acd24927a-1] regionserver.HStore(310): Store=dd4f5e76327a2bdd0e2eb11acd24927a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:24,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened dd4f5e76327a2bdd0e2eb11acd24927a; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61467194, jitterRate=-0.0840674340724945}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-15 06:55:24,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for dd4f5e76327a2bdd0e2eb11acd24927a: 2023-02-15 06:55:24,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a., pid=94, masterSystemTime=1676444063012 2023-02-15 06:55:24,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a. 2023-02-15 06:55:24,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a. 2023-02-15 06:55:24,753 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=dd4f5e76327a2bdd0e2eb11acd24927a, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:24,753 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444124752"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1676444124752"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676444124752"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676444124752"}]},"ts":"1676444124752"} 2023-02-15 06:55:24,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-15 06:55:24,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure dd4f5e76327a2bdd0e2eb11acd24927a, server=jenkins-hbase4.apache.org,35089,1676444032716 in 1 mins, 1.896 sec 2023-02-15 06:55:24,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-15 06:55:24,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a, ASSIGN in 1 mins, 2.055 sec 2023-02-15 06:55:24,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.58 sec 2023-02-15 06:55:24,759 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676444062173","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62586,"client":"172.31.14.131:58654","queuetimems":0,"class":"HMaster"} 2023-02-15 06:55:24,760 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 603, skipping over this response of 3 bytes 2023-02-15 06:55:26,649 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up 2023-02-15 06:55:26,650 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:55:26,652 INFO [Listener at localhost/37669] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:26,652 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:26,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:26,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-15 06:55:26,656 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444126656"}]},"ts":"1676444126656"} 2023-02-15 06:55:26,657 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-15 06:55:26,658 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-15 06:55:26,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a, UNASSIGN}] 2023-02-15 06:55:26,660 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a, UNASSIGN 2023-02-15 06:55:26,660 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=dd4f5e76327a2bdd0e2eb11acd24927a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:26,660 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444126660"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676444126660"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676444126660"}]},"ts":"1676444126660"} 2023-02-15 06:55:26,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure dd4f5e76327a2bdd0e2eb11acd24927a, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:55:26,663 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, UNASSIGN 2023-02-15 06:55:26,664 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=c6d354efa00d71a16486322a96778320, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:26,664 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444126664"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444126664"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444126664"}]},"ts":"1676444126664"} 2023-02-15 06:55:26,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure c6d354efa00d71a16486322a96778320, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:55:26,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-15 06:55:26,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close dd4f5e76327a2bdd0e2eb11acd24927a 2023-02-15 06:55:26,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing dd4f5e76327a2bdd0e2eb11acd24927a, disabling compactions & flushes 2023-02-15 06:55:26,814 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a. 2023-02-15 06:55:26,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a. 2023-02-15 06:55:26,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a. after waiting 0 ms 2023-02-15 06:55:26,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a. 2023-02-15 06:55:26,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:26,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a. 2023-02-15 06:55:26,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for dd4f5e76327a2bdd0e2eb11acd24927a: 2023-02-15 06:55:26,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c6d354efa00d71a16486322a96778320 2023-02-15 06:55:26,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c6d354efa00d71a16486322a96778320, disabling compactions & flushes 2023-02-15 06:55:26,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:55:26,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:55:26,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. after waiting 0 ms 2023-02-15 06:55:26,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:55:26,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed dd4f5e76327a2bdd0e2eb11acd24927a 2023-02-15 06:55:26,818 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=dd4f5e76327a2bdd0e2eb11acd24927a, regionState=CLOSED 2023-02-15 06:55:26,819 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444126818"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676444126818"}]},"ts":"1676444126818"} 2023-02-15 06:55:26,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-15 06:55:26,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-15 06:55:26,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure dd4f5e76327a2bdd0e2eb11acd24927a, server=jenkins-hbase4.apache.org,35089,1676444032716 in 160 msec 2023-02-15 06:55:26,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:26,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320. 2023-02-15 06:55:26,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c6d354efa00d71a16486322a96778320: 2023-02-15 06:55:26,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a, UNASSIGN in 165 msec 2023-02-15 06:55:26,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c6d354efa00d71a16486322a96778320 2023-02-15 06:55:26,826 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=c6d354efa00d71a16486322a96778320, regionState=CLOSED 2023-02-15 06:55:26,826 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676444126826"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444126826"}]},"ts":"1676444126826"} 2023-02-15 06:55:26,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-15 06:55:26,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure c6d354efa00d71a16486322a96778320, server=jenkins-hbase4.apache.org,41759,1676444032818 in 162 msec 2023-02-15 06:55:26,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-15 06:55:26,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=c6d354efa00d71a16486322a96778320, UNASSIGN in 170 msec 2023-02-15 06:55:26,833 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444126833"}]},"ts":"1676444126833"} 2023-02-15 06:55:26,834 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-15 06:55:26,836 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-15 06:55:26,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 185 msec 2023-02-15 06:55:26,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-15 06:55:26,958 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-15 06:55:26,958 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:26,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:26,961 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:26,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-15 06:55:26,961 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:26,966 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320 2023-02-15 06:55:26,968 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/recovered.edits] 2023-02-15 06:55:26,973 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/recovered.edits/7.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320/recovered.edits/7.seqid 2023-02-15 06:55:26,973 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/c6d354efa00d71a16486322a96778320 2023-02-15 06:55:26,973 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-15 06:55:26,976 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:26,978 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-15 06:55:26,980 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-15 06:55:26,981 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:26,981 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-15 06:55:26,981 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444126981"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:26,981 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444126981"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:26,983 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-15 06:55:26,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c6d354efa00d71a16486322a96778320, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676444061665.c6d354efa00d71a16486322a96778320.', STARTKEY => '', ENDKEY => ''}, {ENCODED => dd4f5e76327a2bdd0e2eb11acd24927a, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676444061665_0001.dd4f5e76327a2bdd0e2eb11acd24927a.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-15 06:55:26,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-15 06:55:26,983 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676444126983"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:26,985 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-15 06:55:26,987 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:26,988 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 28 msec 2023-02-15 06:55:27,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-15 06:55:27,063 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-15 06:55:27,075 INFO [Listener at localhost/37669] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=346 (was 411), OpenFileDescriptor=655 (was 842), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=151 (was 252), ProcessCount=168 (was 168), AvailableMemoryMB=3278 (was 3293) 2023-02-15 06:55:27,087 INFO [Listener at localhost/37669] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=347, OpenFileDescriptor=655, MaxFileDescriptor=60000, SystemLoadAverage=151, ProcessCount=168, AvailableMemoryMB=3277 2023-02-15 06:55:27,087 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:55:27,090 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-15 06:55:27,090 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-15 06:55:27,092 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:27,093 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:55:27,094 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:55:27,095 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:27,096 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325 empty. 2023-02-15 06:55:27,096 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:27,096 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-15 06:55:27,105 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-15 06:55:27,106 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0f8b7ddae8d24973658bfe8371be2325, NAME => 'testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:55:27,113 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:27,113 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 0f8b7ddae8d24973658bfe8371be2325, disabling compactions & flushes 2023-02-15 06:55:27,113 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,113 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,113 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. after waiting 0 ms 2023-02-15 06:55:27,113 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,113 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,113 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 0f8b7ddae8d24973658bfe8371be2325: 2023-02-15 06:55:27,115 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:55:27,116 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676444127116"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444127116"}]},"ts":"1676444127116"} 2023-02-15 06:55:27,118 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:55:27,118 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:55:27,118 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444127118"}]},"ts":"1676444127118"} 2023-02-15 06:55:27,119 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-15 06:55:27,128 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:55:27,129 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:55:27,129 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:55:27,129 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:55:27,129 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:55:27,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=0f8b7ddae8d24973658bfe8371be2325, ASSIGN}] 2023-02-15 06:55:27,130 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=0f8b7ddae8d24973658bfe8371be2325, ASSIGN 2023-02-15 06:55:27,131 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=0f8b7ddae8d24973658bfe8371be2325, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:55:27,281 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:55:27,282 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=0f8b7ddae8d24973658bfe8371be2325, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,282 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676444127282"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444127282"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444127282"}]},"ts":"1676444127282"} 2023-02-15 06:55:27,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 0f8b7ddae8d24973658bfe8371be2325, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:55:27,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0f8b7ddae8d24973658bfe8371be2325, NAME => 'testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:55:27,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:55:27,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:27,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:27,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:27,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:27,442 INFO [StoreOpener-0f8b7ddae8d24973658bfe8371be2325-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:27,444 DEBUG [StoreOpener-0f8b7ddae8d24973658bfe8371be2325-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1 2023-02-15 06:55:27,444 DEBUG [StoreOpener-0f8b7ddae8d24973658bfe8371be2325-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1 2023-02-15 06:55:27,445 INFO [StoreOpener-0f8b7ddae8d24973658bfe8371be2325-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f8b7ddae8d24973658bfe8371be2325 columnFamilyName cf1 2023-02-15 06:55:27,445 INFO [StoreOpener-0f8b7ddae8d24973658bfe8371be2325-1] regionserver.HStore(310): Store=0f8b7ddae8d24973658bfe8371be2325/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:27,445 INFO [StoreOpener-0f8b7ddae8d24973658bfe8371be2325-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:27,447 DEBUG [StoreOpener-0f8b7ddae8d24973658bfe8371be2325-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2 2023-02-15 06:55:27,447 DEBUG [StoreOpener-0f8b7ddae8d24973658bfe8371be2325-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2 2023-02-15 06:55:27,448 INFO [StoreOpener-0f8b7ddae8d24973658bfe8371be2325-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f8b7ddae8d24973658bfe8371be2325 columnFamilyName cf2 2023-02-15 06:55:27,448 INFO [StoreOpener-0f8b7ddae8d24973658bfe8371be2325-1] regionserver.HStore(310): Store=0f8b7ddae8d24973658bfe8371be2325/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:27,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:27,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:27,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:55:27,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:27,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:55:27,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0f8b7ddae8d24973658bfe8371be2325; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74790809, jitterRate=0.11446990072727203}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:55:27,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0f8b7ddae8d24973658bfe8371be2325: 2023-02-15 06:55:27,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325., pid=104, masterSystemTime=1676444127436 2023-02-15 06:55:27,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=0f8b7ddae8d24973658bfe8371be2325, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,458 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676444127458"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444127458"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444127458"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444127458"}]},"ts":"1676444127458"} 2023-02-15 06:55:27,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-15 06:55:27,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 0f8b7ddae8d24973658bfe8371be2325, server=jenkins-hbase4.apache.org,41759,1676444032818 in 176 msec 2023-02-15 06:55:27,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-15 06:55:27,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=0f8b7ddae8d24973658bfe8371be2325, ASSIGN in 333 msec 2023-02-15 06:55:27,464 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:55:27,464 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444127464"}]},"ts":"1676444127464"} 2023-02-15 06:55:27,465 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-15 06:55:27,467 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:55:27,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 376 msec 2023-02-15 06:55:27,600 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:27,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-15 06:55:27,602 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-15 06:55:27,602 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,602 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-15 06:55:27,602 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-15 06:55:27,602 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,602 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,603 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,604 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,604 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,603 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,604 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,604 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:27,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:27,605 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:27,605 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:27,605 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,605 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:27,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:27,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:27,608 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:27,608 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:27,608 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. started... 2023-02-15 06:55:27,609 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:27,609 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f8b7ddae8d24973658bfe8371be2325 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-15 06:55:27,611 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,611 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,611 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,611 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:27,612 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,612 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:27,620 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/725db3165b534a36a6993e6583102e7c 2023-02-15 06:55:27,634 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/b977e6432e444e94be92e5246ee69c4b 2023-02-15 06:55:27,637 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/725db3165b534a36a6993e6583102e7c as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/725db3165b534a36a6993e6583102e7c 2023-02-15 06:55:27,641 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/725db3165b534a36a6993e6583102e7c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:55:27,641 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/b977e6432e444e94be92e5246ee69c4b as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/b977e6432e444e94be92e5246ee69c4b 2023-02-15 06:55:27,645 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/b977e6432e444e94be92e5246ee69c4b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:55:27,646 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 0f8b7ddae8d24973658bfe8371be2325 in 37ms, sequenceid=6, compaction requested=false 2023-02-15 06:55:27,646 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-15 06:55:27,646 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f8b7ddae8d24973658bfe8371be2325: 2023-02-15 06:55:27,646 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,646 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:27,646 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:27,646 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,646 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:27,646 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:27,649 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,649 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:27,649 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,649 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:27,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-15 06:55:27,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5173fd39[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:27,651 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-15 06:55:27,651 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,652 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,652 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,652 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,653 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,653 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:27,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,652 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:27,653 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:27,653 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:27,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,653 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:27,653 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:27,653 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:27,653 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:27,653 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:27,653 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:27,654 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:27,654 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:27,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:27,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:27,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:27,655 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:27,655 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:27,655 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:27,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:27,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:27,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:27,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-15 06:55:27,658 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-15 06:55:27,658 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:27,658 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:27,658 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-15 06:55:27,658 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:27,660 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,660 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,660 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,660 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,660 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,661 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,660 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:27,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:27,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,661 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:27,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,661 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,662 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:27,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:27,664 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,664 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,664 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,667 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-15 06:55:27,667 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:27,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,667 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:27,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,667 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:27,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,667 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,668 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:27,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,768 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:27,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:27,780 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:27,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:27,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-15 06:55:27,782 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-15 06:55:27,782 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,782 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-15 06:55:27,782 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-15 06:55:27,782 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,782 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,783 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,783 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,783 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,783 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,784 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,784 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:27,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,784 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,784 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,784 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:27,785 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:27,785 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:27,785 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:27,785 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,785 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:27,785 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,785 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:27,785 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:27,785 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:27,785 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:27,785 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:27,785 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:27,786 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:27,786 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:27,786 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:27,786 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:27,786 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,786 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:27,786 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:27,786 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:27,786 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,786 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. started... 2023-02-15 06:55:27,786 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f8b7ddae8d24973658bfe8371be2325 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:27,787 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,791 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,791 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,792 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:27,792 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,792 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:27,799 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/e9ecb5ad36c549c7b49a1e8758321ef6 2023-02-15 06:55:27,810 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/37c87836a48d405d8cc01eb8528e837e 2023-02-15 06:55:27,813 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/e9ecb5ad36c549c7b49a1e8758321ef6 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/e9ecb5ad36c549c7b49a1e8758321ef6 2023-02-15 06:55:27,817 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/e9ecb5ad36c549c7b49a1e8758321ef6, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:55:27,818 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/37c87836a48d405d8cc01eb8528e837e as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/37c87836a48d405d8cc01eb8528e837e 2023-02-15 06:55:27,821 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/37c87836a48d405d8cc01eb8528e837e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:55:27,822 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f8b7ddae8d24973658bfe8371be2325 in 36ms, sequenceid=19, compaction requested=false 2023-02-15 06:55:27,822 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f8b7ddae8d24973658bfe8371be2325: 2023-02-15 06:55:27,822 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:27,824 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,824 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:27,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:27,824 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,824 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:27,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:27,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:27,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:27,826 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-15 06:55:27,826 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@755a7960[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:27,826 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-15 06:55:27,826 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,827 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,827 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,827 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:27,827 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:27,827 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:27,827 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,827 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:27,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,827 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:27,827 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:27,828 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,828 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:27,828 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:27,828 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:27,827 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:27,828 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:27,828 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:27,829 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:27,832 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:27,832 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:27,832 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:27,833 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:27,833 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:27,833 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,833 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:27,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:27,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:27,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:27,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:27,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:27,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:27,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-15 06:55:27,836 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-15 06:55:27,836 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:27,836 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:27,836 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-15 06:55:27,836 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:27,837 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,837 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:27,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:27,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,837 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,837 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,837 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,838 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,838 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,838 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,838 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,839 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:27,840 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:27,840 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,841 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,843 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,843 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,843 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,843 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,843 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,843 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,843 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-15 06:55:27,843 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:27,843 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,843 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,843 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:27,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,843 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,844 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:27,844 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,843 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:27,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,844 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,844 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:27,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,844 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,844 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:27,844 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:27,844 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:27,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:27,845 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,944 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:27,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:27,958 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:27,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:27,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-15 06:55:27,959 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-15 06:55:27,959 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,959 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-15 06:55:27,959 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-15 06:55:27,960 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,960 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,962 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,962 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,962 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,962 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:27,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:27,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,962 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,962 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:27,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:27,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:27,964 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:27,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:27,964 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,964 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:27,964 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:27,964 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:27,965 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:27,964 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:27,965 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:27,965 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:27,965 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:27,965 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:27,965 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:27,965 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:27,966 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:27,966 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:27,966 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,966 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. started... 2023-02-15 06:55:27,966 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f8b7ddae8d24973658bfe8371be2325 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:27,966 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,967 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,967 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,967 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:27,968 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:27,968 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:27,975 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/7060ed44ad1545a9b3bccb60d2ebeebb 2023-02-15 06:55:27,986 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/a98cc2c8ee4c4fd5aafc1ae473869991 2023-02-15 06:55:27,990 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/7060ed44ad1545a9b3bccb60d2ebeebb as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/7060ed44ad1545a9b3bccb60d2ebeebb 2023-02-15 06:55:27,993 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/7060ed44ad1545a9b3bccb60d2ebeebb, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:55:27,994 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/a98cc2c8ee4c4fd5aafc1ae473869991 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/a98cc2c8ee4c4fd5aafc1ae473869991 2023-02-15 06:55:27,998 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/a98cc2c8ee4c4fd5aafc1ae473869991, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:55:27,999 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f8b7ddae8d24973658bfe8371be2325 in 33ms, sequenceid=32, compaction requested=true 2023-02-15 06:55:27,999 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f8b7ddae8d24973658bfe8371be2325: 2023-02-15 06:55:27,999 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:27,999 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:27,999 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:27,999 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:27,999 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:27,999 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:28,001 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,001 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,001 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,002 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:28,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-15 06:55:28,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ca64d08[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:28,004 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-15 06:55:28,004 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,011 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,011 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,011 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,011 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,011 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,011 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:28,011 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,011 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:28,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,011 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,011 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,011 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,011 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,011 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:28,011 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:28,012 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,012 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,013 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,013 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,014 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,014 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,014 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,015 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,016 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,016 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:28,019 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-15 06:55:28,019 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-15 06:55:28,019 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:28,019 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:28,019 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-15 06:55:28,019 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:28,021 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,021 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,021 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,021 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,021 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,021 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,022 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,022 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,022 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,023 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,024 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,025 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,025 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,036 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-15 06:55:28,036 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:28,036 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,037 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:28,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,037 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:28,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,037 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,037 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,037 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,037 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,037 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,037 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,037 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,137 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:28,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:28,150 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:28,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:28,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-15 06:55:28,151 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-15 06:55:28,151 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,151 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-15 06:55:28,151 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-15 06:55:28,152 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,152 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,153 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,153 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,153 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,153 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,154 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,154 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:28,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:28,155 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:28,155 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:28,155 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:28,155 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,155 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:28,155 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,155 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:28,155 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:28,155 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:28,155 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:28,156 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:28,156 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:28,156 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:28,155 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:28,156 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:28,156 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,156 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:28,156 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:28,156 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:28,156 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:28,156 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. started... 2023-02-15 06:55:28,156 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f8b7ddae8d24973658bfe8371be2325 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:28,157 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,157 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,157 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,157 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,157 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:28,157 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:28,165 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/bcabd9a61fe74897baadeca268cfd7dd 2023-02-15 06:55:28,179 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/c042e380afef4f1e95fc09a359bad088 2023-02-15 06:55:28,183 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/bcabd9a61fe74897baadeca268cfd7dd as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/bcabd9a61fe74897baadeca268cfd7dd 2023-02-15 06:55:28,186 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/bcabd9a61fe74897baadeca268cfd7dd, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:55:28,187 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/c042e380afef4f1e95fc09a359bad088 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/c042e380afef4f1e95fc09a359bad088 2023-02-15 06:55:28,191 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/c042e380afef4f1e95fc09a359bad088, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:55:28,192 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f8b7ddae8d24973658bfe8371be2325 in 36ms, sequenceid=45, compaction requested=true 2023-02-15 06:55:28,192 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f8b7ddae8d24973658bfe8371be2325: 2023-02-15 06:55:28,192 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:28,192 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:28,192 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:28,192 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,192 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:28,192 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:28,195 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,195 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,195 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,195 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:28,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-15 06:55:28,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20a754fa[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:28,197 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-15 06:55:28,197 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,198 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,198 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,198 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,198 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,199 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,199 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:28,200 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,200 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,200 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,201 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,201 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,201 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,201 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,201 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,201 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,201 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:28,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-15 06:55:28,204 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-15 06:55:28,204 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:28,204 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:28,204 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-15 06:55:28,205 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:28,207 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,207 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,207 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,207 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,207 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,207 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,208 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,208 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,208 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,209 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,210 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,211 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,211 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,213 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,213 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,213 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,214 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,214 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,214 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,214 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,214 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,214 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,214 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,214 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,214 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,214 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-15 06:55:28,214 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:28,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,214 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,215 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,215 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:28,215 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,215 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:28,215 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,215 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,215 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,315 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:28,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:28,328 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:28,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:28,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-15 06:55:28,330 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-15 06:55:28,330 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,330 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-15 06:55:28,330 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-15 06:55:28,330 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,330 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,331 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,332 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,332 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,332 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,332 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,332 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:28,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:28,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,333 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:28,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:28,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,333 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:28,333 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:28,333 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:28,333 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:28,333 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,333 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:28,333 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:28,334 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:28,334 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:28,334 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:28,334 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,334 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:28,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:28,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,334 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:28,334 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:28,334 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:28,334 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:28,334 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:28,334 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:28,334 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:28,334 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. started... 2023-02-15 06:55:28,334 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:28,334 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,334 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f8b7ddae8d24973658bfe8371be2325 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:28,334 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:28,335 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:28,335 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:28,335 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,338 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,338 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:28,338 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,338 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,338 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:28,344 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/35da1f9e4bdc4c3a964fe43a5cbbc4a3 2023-02-15 06:55:28,356 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/8d0a0c7984714baab2aca5a30919131d 2023-02-15 06:55:28,361 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/35da1f9e4bdc4c3a964fe43a5cbbc4a3 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/35da1f9e4bdc4c3a964fe43a5cbbc4a3 2023-02-15 06:55:28,364 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/35da1f9e4bdc4c3a964fe43a5cbbc4a3, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:55:28,365 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/8d0a0c7984714baab2aca5a30919131d as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/8d0a0c7984714baab2aca5a30919131d 2023-02-15 06:55:28,369 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/8d0a0c7984714baab2aca5a30919131d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:55:28,369 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f8b7ddae8d24973658bfe8371be2325 in 35ms, sequenceid=58, compaction requested=true 2023-02-15 06:55:28,370 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f8b7ddae8d24973658bfe8371be2325: 2023-02-15 06:55:28,370 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:28,370 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:28,370 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:28,370 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,370 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:28,370 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:28,371 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,371 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,372 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,372 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:28,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-15 06:55:28,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ef3efe2[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:28,374 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-15 06:55:28,374 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,375 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,375 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,375 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,375 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,375 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,375 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,375 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:28,375 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,375 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,375 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:28,375 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:28,375 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,375 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,375 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:28,380 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,380 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,381 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,380 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,381 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,381 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,383 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,383 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,383 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,383 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:28,386 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-15 06:55:28,386 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-15 06:55:28,387 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:28,387 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:28,387 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-15 06:55:28,387 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:28,388 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,388 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,388 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,388 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,388 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,389 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,389 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,389 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,389 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,390 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,392 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,392 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,392 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,395 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-15 06:55:28,395 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:28,395 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,396 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:28,396 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,395 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,396 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,396 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:28,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,396 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,396 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,396 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,397 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,496 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:28,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:28,509 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:28,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:28,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-15 06:55:28,510 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-15 06:55:28,510 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,510 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-15 06:55:28,510 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-15 06:55:28,511 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,511 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,513 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,513 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,513 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,513 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,514 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,514 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:28,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-15 06:55:28,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:28,515 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:28,515 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:28,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:28,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-15 06:55:28,515 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:28,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:28,516 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:28,516 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:28,516 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. started... 2023-02-15 06:55:28,517 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0f8b7ddae8d24973658bfe8371be2325 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:28,517 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,518 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,518 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,518 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:28,518 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,518 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:28,551 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/daa1ab10bfc84ad4a573712c9ded3aea 2023-02-15 06:55:28,562 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/6296f1c4dc56474d9e6ae7d11cded471 2023-02-15 06:55:28,565 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/daa1ab10bfc84ad4a573712c9ded3aea as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/daa1ab10bfc84ad4a573712c9ded3aea 2023-02-15 06:55:28,569 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/daa1ab10bfc84ad4a573712c9ded3aea, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:55:28,570 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/6296f1c4dc56474d9e6ae7d11cded471 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/6296f1c4dc56474d9e6ae7d11cded471 2023-02-15 06:55:28,573 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/6296f1c4dc56474d9e6ae7d11cded471, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:55:28,574 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0f8b7ddae8d24973658bfe8371be2325 in 57ms, sequenceid=71, compaction requested=true 2023-02-15 06:55:28,574 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0f8b7ddae8d24973658bfe8371be2325: 2023-02-15 06:55:28,574 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:28,574 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:28,574 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:28,574 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,574 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-15 06:55:28,574 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-15 06:55:28,576 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,576 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,576 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,576 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:28,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-15 06:55:28,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48440a25[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:28,578 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-15 06:55:28,578 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,581 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,580 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,581 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:28,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,581 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,581 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-15 06:55:28,581 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,581 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,581 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:28,581 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:28,581 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-15 06:55:28,581 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:28,582 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,582 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,582 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,583 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,583 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,583 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-15 06:55:28,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,583 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,583 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,583 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,583 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-15 06:55:28,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:28,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-15 06:55:28,586 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-15 06:55:28,586 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:28,586 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:28,586 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-15 06:55:28,586 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:28,587 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,587 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,588 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,588 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,588 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,588 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,587 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,588 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:28,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:28,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:28,588 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,589 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:28,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,590 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:28,591 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-15 06:55:28,591 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,595 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,595 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-15 06:55:28,596 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:28,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:28,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,596 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,595 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:28,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,596 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:28,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:28,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:28,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,596 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:28,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:28,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:28,696 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:28,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:28,704 INFO [Listener at localhost/37669] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,706 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.StoreUtils(129): cannot split hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/e9ecb5ad36c549c7b49a1e8758321ef6 because midkey is the same as first or last row 2023-02-15 06:55:28,706 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.StoreUtils(129): cannot split hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/37c87836a48d405d8cc01eb8528e837e because midkey is the same as first or last row 2023-02-15 06:55:28,706 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.HRegion(2744): Flushing 0f8b7ddae8d24973658bfe8371be2325 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-15 06:55:28,715 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/83b8d4be89f7413ebc98d62ea704de2a 2023-02-15 06:55:28,725 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/958a044a89144f32bed70d91082cfa66 2023-02-15 06:55:28,729 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf1/83b8d4be89f7413ebc98d62ea704de2a as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/83b8d4be89f7413ebc98d62ea704de2a 2023-02-15 06:55:28,732 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/83b8d4be89f7413ebc98d62ea704de2a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:55:28,733 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/.tmp/cf2/958a044a89144f32bed70d91082cfa66 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/958a044a89144f32bed70d91082cfa66 2023-02-15 06:55:28,737 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/958a044a89144f32bed70d91082cfa66, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:55:28,737 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 0f8b7ddae8d24973658bfe8371be2325 in 31ms, sequenceid=82, compaction requested=true 2023-02-15 06:55:28,738 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.HRegion(2445): Flush status journal for 0f8b7ddae8d24973658bfe8371be2325: 2023-02-15 06:55:28,738 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.StoreUtils(129): cannot split hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/e9ecb5ad36c549c7b49a1e8758321ef6 because midkey is the same as first or last row 2023-02-15 06:55:28,739 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41759] regionserver.StoreUtils(129): cannot split hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/37c87836a48d405d8cc01eb8528e837e because midkey is the same as first or last row 2023-02-15 06:55:28,739 DEBUG [Listener at localhost/37669] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,739 DEBUG [Listener at localhost/37669] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-15 06:55:28,739 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:55:28,740 INFO [Listener at localhost/37669] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-15 06:55:28,741 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-15 06:55:28,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:28,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-15 06:55:28,744 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444128744"}]},"ts":"1676444128744"} 2023-02-15 06:55:28,745 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-15 06:55:28,746 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-15 06:55:28,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=0f8b7ddae8d24973658bfe8371be2325, UNASSIGN}] 2023-02-15 06:55:28,747 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=0f8b7ddae8d24973658bfe8371be2325, UNASSIGN 2023-02-15 06:55:28,748 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=0f8b7ddae8d24973658bfe8371be2325, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:28,748 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676444128748"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444128748"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444128748"}]},"ts":"1676444128748"} 2023-02-15 06:55:28,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 0f8b7ddae8d24973658bfe8371be2325, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:55:28,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-15 06:55:28,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:28,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0f8b7ddae8d24973658bfe8371be2325, disabling compactions & flushes 2023-02-15 06:55:28,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:28,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:28,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. after waiting 0 ms 2023-02-15 06:55:28,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:28,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-15 06:55:28,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:28,905 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325. 2023-02-15 06:55:28,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0f8b7ddae8d24973658bfe8371be2325: 2023-02-15 06:55:28,907 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:28,907 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=0f8b7ddae8d24973658bfe8371be2325, regionState=CLOSED 2023-02-15 06:55:28,907 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676444128907"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444128907"}]},"ts":"1676444128907"} 2023-02-15 06:55:28,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-15 06:55:28,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 0f8b7ddae8d24973658bfe8371be2325, server=jenkins-hbase4.apache.org,41759,1676444032818 in 159 msec 2023-02-15 06:55:28,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-15 06:55:28,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=0f8b7ddae8d24973658bfe8371be2325, UNASSIGN in 162 msec 2023-02-15 06:55:28,911 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444128911"}]},"ts":"1676444128911"} 2023-02-15 06:55:28,912 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-15 06:55:28,914 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-15 06:55:28,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 173 msec 2023-02-15 06:55:29,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-15 06:55:29,046 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-15 06:55:29,047 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-15 06:55:29,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:29,049 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:29,050 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:29,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-15 06:55:29,053 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:29,054 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/recovered.edits] 2023-02-15 06:55:29,057 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/35da1f9e4bdc4c3a964fe43a5cbbc4a3 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/35da1f9e4bdc4c3a964fe43a5cbbc4a3 2023-02-15 06:55:29,058 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/7060ed44ad1545a9b3bccb60d2ebeebb to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/7060ed44ad1545a9b3bccb60d2ebeebb 2023-02-15 06:55:29,059 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/725db3165b534a36a6993e6583102e7c to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/725db3165b534a36a6993e6583102e7c 2023-02-15 06:55:29,060 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/83b8d4be89f7413ebc98d62ea704de2a to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/83b8d4be89f7413ebc98d62ea704de2a 2023-02-15 06:55:29,061 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/bcabd9a61fe74897baadeca268cfd7dd to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/bcabd9a61fe74897baadeca268cfd7dd 2023-02-15 06:55:29,062 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/daa1ab10bfc84ad4a573712c9ded3aea to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/daa1ab10bfc84ad4a573712c9ded3aea 2023-02-15 06:55:29,063 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/e9ecb5ad36c549c7b49a1e8758321ef6 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf1/e9ecb5ad36c549c7b49a1e8758321ef6 2023-02-15 06:55:29,065 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/37c87836a48d405d8cc01eb8528e837e to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/37c87836a48d405d8cc01eb8528e837e 2023-02-15 06:55:29,066 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/6296f1c4dc56474d9e6ae7d11cded471 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/6296f1c4dc56474d9e6ae7d11cded471 2023-02-15 06:55:29,067 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/8d0a0c7984714baab2aca5a30919131d to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/8d0a0c7984714baab2aca5a30919131d 2023-02-15 06:55:29,068 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/958a044a89144f32bed70d91082cfa66 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/958a044a89144f32bed70d91082cfa66 2023-02-15 06:55:29,069 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/a98cc2c8ee4c4fd5aafc1ae473869991 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/a98cc2c8ee4c4fd5aafc1ae473869991 2023-02-15 06:55:29,070 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/b977e6432e444e94be92e5246ee69c4b to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/b977e6432e444e94be92e5246ee69c4b 2023-02-15 06:55:29,071 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/c042e380afef4f1e95fc09a359bad088 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/cf2/c042e380afef4f1e95fc09a359bad088 2023-02-15 06:55:29,074 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/recovered.edits/85.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325/recovered.edits/85.seqid 2023-02-15 06:55:29,074 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testInvalidSplitKey/0f8b7ddae8d24973658bfe8371be2325 2023-02-15 06:55:29,074 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-15 06:55:29,076 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:29,077 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-15 06:55:29,081 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-15 06:55:29,082 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:29,082 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-15 06:55:29,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444129082"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:29,083 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-15 06:55:29,083 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0f8b7ddae8d24973658bfe8371be2325, NAME => 'testInvalidSplitKey,,1676444127091.0f8b7ddae8d24973658bfe8371be2325.', STARTKEY => '', ENDKEY => ''}] 2023-02-15 06:55:29,083 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-15 06:55:29,083 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676444129083"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:29,084 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-15 06:55:29,087 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:29,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 40 msec 2023-02-15 06:55:29,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-15 06:55:29,152 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-15 06:55:29,165 INFO [Listener at localhost/37669] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=369 (was 347) - Thread LEAK? -, OpenFileDescriptor=744 (was 655) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=151 (was 151), ProcessCount=168 (was 168), AvailableMemoryMB=3209 (was 3277) 2023-02-15 06:55:29,178 INFO [Listener at localhost/37669] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=369, OpenFileDescriptor=744, MaxFileDescriptor=60000, SystemLoadAverage=151, ProcessCount=168, AvailableMemoryMB=3209 2023-02-15 06:55:29,178 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:55:29,181 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-15 06:55:29,181 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-15 06:55:29,182 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:29,183 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:55:29,184 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:55:29,185 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:29,186 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0 empty. 2023-02-15 06:55:29,186 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:29,186 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-15 06:55:29,196 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-15 06:55:29,197 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => d29d87088bdb1ecf07d47a3df3023ac0, NAME => 'testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:55:29,204 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:29,204 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing d29d87088bdb1ecf07d47a3df3023ac0, disabling compactions & flushes 2023-02-15 06:55:29,204 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,204 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,204 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. after waiting 0 ms 2023-02-15 06:55:29,204 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,204 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,204 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for d29d87088bdb1ecf07d47a3df3023ac0: 2023-02-15 06:55:29,206 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:55:29,206 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444129206"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444129206"}]},"ts":"1676444129206"} 2023-02-15 06:55:29,207 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:55:29,208 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:55:29,208 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444129208"}]},"ts":"1676444129208"} 2023-02-15 06:55:29,209 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-15 06:55:29,212 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:55:29,213 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:55:29,213 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:55:29,213 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:55:29,213 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:55:29,213 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, ASSIGN}] 2023-02-15 06:55:29,214 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, ASSIGN 2023-02-15 06:55:29,215 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40475,1676444032771; forceNewPlan=false, retain=false 2023-02-15 06:55:29,365 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:55:29,366 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=d29d87088bdb1ecf07d47a3df3023ac0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,366 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444129366"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444129366"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444129366"}]},"ts":"1676444129366"} 2023-02-15 06:55:29,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:55:29,379 ERROR [BP-145338253-172.31.14.131-1676444028112 heartbeating to localhost/127.0.0.1:41175] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-145338253-172.31.14.131-1676444028112 (Datanode Uuid 36321d6c-46ca-442e-92ed-6bcb61ed715a) service to localhost/127.0.0.1:41175 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-15 06:55:29,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d29d87088bdb1ecf07d47a3df3023ac0, NAME => 'testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:55:29,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:55:29,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:29,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:29,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:29,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:29,526 INFO [StoreOpener-d29d87088bdb1ecf07d47a3df3023ac0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:29,527 DEBUG [StoreOpener-d29d87088bdb1ecf07d47a3df3023ac0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1 2023-02-15 06:55:29,527 DEBUG [StoreOpener-d29d87088bdb1ecf07d47a3df3023ac0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1 2023-02-15 06:55:29,528 INFO [StoreOpener-d29d87088bdb1ecf07d47a3df3023ac0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d29d87088bdb1ecf07d47a3df3023ac0 columnFamilyName cf1 2023-02-15 06:55:29,528 INFO [StoreOpener-d29d87088bdb1ecf07d47a3df3023ac0-1] regionserver.HStore(310): Store=d29d87088bdb1ecf07d47a3df3023ac0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:29,528 INFO [StoreOpener-d29d87088bdb1ecf07d47a3df3023ac0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:29,529 DEBUG [StoreOpener-d29d87088bdb1ecf07d47a3df3023ac0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2 2023-02-15 06:55:29,529 DEBUG [StoreOpener-d29d87088bdb1ecf07d47a3df3023ac0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2 2023-02-15 06:55:29,530 INFO [StoreOpener-d29d87088bdb1ecf07d47a3df3023ac0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d29d87088bdb1ecf07d47a3df3023ac0 columnFamilyName cf2 2023-02-15 06:55:29,530 INFO [StoreOpener-d29d87088bdb1ecf07d47a3df3023ac0-1] regionserver.HStore(310): Store=d29d87088bdb1ecf07d47a3df3023ac0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:29,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:29,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:29,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:55:29,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:29,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:55:29,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d29d87088bdb1ecf07d47a3df3023ac0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67941382, jitterRate=0.012405484914779663}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:55:29,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d29d87088bdb1ecf07d47a3df3023ac0: 2023-02-15 06:55:29,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0., pid=111, masterSystemTime=1676444129520 2023-02-15 06:55:29,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,538 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=d29d87088bdb1ecf07d47a3df3023ac0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,538 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444129538"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444129538"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444129538"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444129538"}]},"ts":"1676444129538"} 2023-02-15 06:55:29,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-15 06:55:29,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 in 171 msec 2023-02-15 06:55:29,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-15 06:55:29,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, ASSIGN in 328 msec 2023-02-15 06:55:29,543 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:55:29,543 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444129543"}]},"ts":"1676444129543"} 2023-02-15 06:55:29,544 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-15 06:55:29,547 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:55:29,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 366 msec 2023-02-15 06:55:29,689 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:29,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-15 06:55:29,690 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:29,690 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:29,690 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-15 06:55:29,690 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:55:29,691 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,691 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,692 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,692 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,693 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,693 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:29,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:29,694 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:29,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:29,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,694 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:29,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:29,694 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:29,695 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,696 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,696 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:29,700 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:29,700 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:29,700 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:29,700 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:29,700 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:29,700 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:29,700 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:29,700 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:29,700 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:29,700 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:29,700 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:29,700 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:29,700 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,700 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. started... 2023-02-15 06:55:29,703 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d29d87088bdb1ecf07d47a3df3023ac0 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-15 06:55:29,704 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,704 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,704 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:29,718 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291 2023-02-15 06:55:29,739 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/0ac3e26234464b63a22bd50772d242dd 2023-02-15 06:55:29,743 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291 2023-02-15 06:55:29,747 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:55:29,748 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/0ac3e26234464b63a22bd50772d242dd as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd 2023-02-15 06:55:29,751 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:55:29,752 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d29d87088bdb1ecf07d47a3df3023ac0 in 49ms, sequenceid=6, compaction requested=false 2023-02-15 06:55:29,752 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:29,753 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d29d87088bdb1ecf07d47a3df3023ac0: 2023-02-15 06:55:29,753 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,753 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:29,753 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:29,753 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,753 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:29,753 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:29,756 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:29,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-15 06:55:29,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4604c028[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:29,758 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:55:29,758 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,760 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,760 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,760 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:29,760 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,760 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:29,761 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:29,762 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:29,762 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:29,763 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:29,763 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:29,763 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:29,763 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,763 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:29,763 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:29,763 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:55:29,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:55:29,766 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-15 06:55:29,766 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:29,767 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:29,767 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-15 06:55:29,767 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:29,769 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,769 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,769 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,769 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,769 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,769 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:29,770 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,770 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,770 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,771 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:29,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,772 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:29,772 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,773 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,776 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:29,776 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:29,776 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:29,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,776 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,777 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,777 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:29,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,777 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,777 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:29,777 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,777 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,777 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,777 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,877 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:29,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:29,888 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:29,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:29,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-15 06:55:29,890 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:29,890 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:29,890 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-15 06:55:29,890 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:55:29,890 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,890 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,891 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,891 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,891 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,891 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,892 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,892 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:29,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,892 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,892 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:29,893 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:29,893 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:29,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:29,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:29,893 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:29,893 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:29,893 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:29,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:29,894 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:29,894 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:29,894 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:29,894 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:29,894 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:29,894 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:29,894 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:29,894 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:29,895 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:29,895 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:29,895 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,895 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. started... 2023-02-15 06:55:29,895 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,895 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d29d87088bdb1ecf07d47a3df3023ac0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:29,897 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,897 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,897 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:29,897 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,897 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:29,903 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/5075082840b04a81acd1f17d868d45bb 2023-02-15 06:55:29,918 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/7eec12b2504446f8b0d0bb81d497dd9e 2023-02-15 06:55:29,924 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/5075082840b04a81acd1f17d868d45bb as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb 2023-02-15 06:55:29,928 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:55:29,929 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/7eec12b2504446f8b0d0bb81d497dd9e as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e 2023-02-15 06:55:29,932 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:55:29,933 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d29d87088bdb1ecf07d47a3df3023ac0 in 38ms, sequenceid=19, compaction requested=false 2023-02-15 06:55:29,933 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d29d87088bdb1ecf07d47a3df3023ac0: 2023-02-15 06:55:29,933 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:29,933 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:29,933 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:29,933 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,934 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:29,934 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:29,941 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,941 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:29,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:29,941 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,941 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:29,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:29,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:29,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-15 06:55:29,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c9e5c96[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:29,943 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:55:29,943 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,945 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,945 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,945 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,945 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:29,945 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,945 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:29,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,945 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:29,945 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:29,945 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:29,945 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:29,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,945 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:29,946 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:29,946 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:29,946 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:29,947 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:29,947 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,947 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:29,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:29,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:29,947 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:29,948 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:29,949 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:29,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:29,949 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:29,950 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:29,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:29,950 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:29,950 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:29,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:29,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:55:29,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:55:29,953 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-15 06:55:29,953 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:29,953 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:29,953 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-15 06:55:29,953 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:29,954 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,954 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:29,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,954 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,954 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,954 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,955 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:29,955 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,955 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:29,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,956 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,956 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:29,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:29,958 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,958 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,959 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,961 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,961 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,961 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,961 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,961 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,962 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:29,962 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:29,962 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:29,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,961 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,962 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,962 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,962 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:29,962 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,961 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,961 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:29,962 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:29,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,962 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:29,962 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:29,962 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:29,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:29,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:29,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:29,963 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,963 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:29,963 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:29,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:29,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,063 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:30,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:30,075 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:30,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:30,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-15 06:55:30,076 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:30,076 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,076 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-15 06:55:30,076 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:55:30,076 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,077 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,078 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,078 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,078 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,078 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,079 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,079 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,079 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,079 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,080 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,080 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,080 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,080 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,080 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,080 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,081 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:30,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,081 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. started... 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,081 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d29d87088bdb1ecf07d47a3df3023ac0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,081 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,092 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/eca3f849060c4dc2bdf9df512c78d058 2023-02-15 06:55:30,103 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/09cdca57f96541bfb09f40e41376dedf 2023-02-15 06:55:30,107 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/eca3f849060c4dc2bdf9df512c78d058 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058 2023-02-15 06:55:30,110 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:55:30,111 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/09cdca57f96541bfb09f40e41376dedf as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf 2023-02-15 06:55:30,114 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:55:30,115 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d29d87088bdb1ecf07d47a3df3023ac0 in 34ms, sequenceid=32, compaction requested=true 2023-02-15 06:55:30,115 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d29d87088bdb1ecf07d47a3df3023ac0: 2023-02-15 06:55:30,115 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:30,115 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:30,115 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:30,115 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,115 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,115 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,117 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,117 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,117 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,117 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-15 06:55:30,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2de32418[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:30,119 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:55:30,119 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,119 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,119 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,120 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,120 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,120 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:30,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,120 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:30,122 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,122 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,122 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,123 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,123 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,123 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,123 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,123 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,123 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,123 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:55:30,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:55:30,126 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-15 06:55:30,126 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:30,126 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:30,126 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,126 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:30,128 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,128 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,128 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,128 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,128 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,128 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,128 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,128 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,129 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,129 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,131 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,131 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,131 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,141 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:30,141 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:30,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,142 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:30,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,141 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,141 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,142 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:30,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,142 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,142 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,142 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,142 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,142 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,142 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,142 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,143 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,242 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:30,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:30,254 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:30,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:30,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-15 06:55:30,255 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:30,255 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,255 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-15 06:55:30,255 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:55:30,256 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,256 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,257 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,257 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,257 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,257 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,257 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,258 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:30,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,259 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:30,260 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,260 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,260 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:30,260 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. started... 2023-02-15 06:55:30,260 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d29d87088bdb1ecf07d47a3df3023ac0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:30,261 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,261 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,261 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,261 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,262 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,262 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,268 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/645b51fa33e848b29c29942ee2c36ac3 2023-02-15 06:55:30,279 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/0ee22f032c2e41f486e4a61998c71148 2023-02-15 06:55:30,283 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/645b51fa33e848b29c29942ee2c36ac3 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3 2023-02-15 06:55:30,286 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:55:30,287 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/0ee22f032c2e41f486e4a61998c71148 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148 2023-02-15 06:55:30,290 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:55:30,291 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d29d87088bdb1ecf07d47a3df3023ac0 in 31ms, sequenceid=45, compaction requested=true 2023-02-15 06:55:30,291 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d29d87088bdb1ecf07d47a3df3023ac0: 2023-02-15 06:55:30,291 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:30,291 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:30,291 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:30,291 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,291 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,291 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,294 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,294 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,295 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,295 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-15 06:55:30,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@459e06a3[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:30,296 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:55:30,296 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,297 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,297 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,297 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,297 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,298 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:30,297 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,298 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,298 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:30,298 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,298 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,298 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:30,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,298 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,298 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,298 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:30,299 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,299 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,299 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,300 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,300 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,300 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,301 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,301 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,302 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,302 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:55:30,304 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:55:30,304 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-15 06:55:30,304 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:30,304 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:30,305 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,305 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:30,306 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,306 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,306 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,306 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,307 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,307 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,309 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,309 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,309 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,312 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,312 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:30,312 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,313 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,312 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,313 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,313 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:30,313 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,313 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,313 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,313 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,413 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:30,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:30,425 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:30,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:30,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-15 06:55:30,427 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:30,427 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,427 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-15 06:55:30,427 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:55:30,427 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,427 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,429 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,429 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,429 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,429 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,429 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,429 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:30,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,430 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,430 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,431 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,431 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,431 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,431 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,431 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:30,431 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:30,431 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:30,431 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. started... 2023-02-15 06:55:30,431 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d29d87088bdb1ecf07d47a3df3023ac0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:30,433 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,434 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,434 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,434 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,434 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,434 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,440 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/06fcad3fc84d487ca90f6f5ec9c98860 2023-02-15 06:55:30,450 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/e2cd97c6613848c9b1b11b7017f2981b 2023-02-15 06:55:30,454 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/06fcad3fc84d487ca90f6f5ec9c98860 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860 2023-02-15 06:55:30,458 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:55:30,459 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/e2cd97c6613848c9b1b11b7017f2981b as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b 2023-02-15 06:55:30,462 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:55:30,463 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d29d87088bdb1ecf07d47a3df3023ac0 in 32ms, sequenceid=58, compaction requested=true 2023-02-15 06:55:30,463 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d29d87088bdb1ecf07d47a3df3023ac0: 2023-02-15 06:55:30,463 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:30,463 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:30,463 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:30,463 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,463 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,463 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,467 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,467 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,467 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,467 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-15 06:55:30,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@257b9bb8[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:30,469 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:55:30,469 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,473 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,473 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,473 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,473 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,473 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,473 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:30,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,473 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,474 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,474 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,474 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:30,473 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,474 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,474 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:30,474 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:30,475 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,475 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,475 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,476 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,476 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,476 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,476 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,476 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,476 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,476 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:55:30,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:55:30,479 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-15 06:55:30,479 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:30,479 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:30,479 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,479 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:30,481 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,481 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,481 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,482 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,482 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,484 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,484 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,485 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,488 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,488 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,488 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,488 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,488 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,488 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,488 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,488 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,488 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,488 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:30,489 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:30,488 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,489 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,489 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,489 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,489 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:30,489 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,490 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,489 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:30,490 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,490 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,490 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,590 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:30,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:30,602 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:30,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:30,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-15 06:55:30,603 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:30,603 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,603 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-15 06:55:30,603 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-15 06:55:30,604 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,604 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,605 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,605 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,605 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,605 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,605 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,606 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,606 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-15 06:55:30,606 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,606 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,607 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,607 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,607 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:30,607 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:30,607 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,607 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. started... 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:30,607 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d29d87088bdb1ecf07d47a3df3023ac0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,608 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,609 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,609 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,609 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,610 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,610 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,615 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/15c746103b024fb587cc8160cac4fa9e 2023-02-15 06:55:30,626 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/deae548396ce40b6987ba172e3de406b 2023-02-15 06:55:30,630 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/15c746103b024fb587cc8160cac4fa9e as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e 2023-02-15 06:55:30,633 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:55:30,634 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/deae548396ce40b6987ba172e3de406b as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b 2023-02-15 06:55:30,638 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:55:30,638 INFO [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d29d87088bdb1ecf07d47a3df3023ac0 in 31ms, sequenceid=71, compaction requested=true 2023-02-15 06:55:30,638 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d29d87088bdb1ecf07d47a3df3023ac0: 2023-02-15 06:55:30,638 DEBUG [rs(jenkins-hbase4.apache.org,40475,1676444032771)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:30,639 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:30,639 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:30,639 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,639 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-15 06:55:30,639 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-15 06:55:30,645 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,645 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,645 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,645 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:30,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-15 06:55:30,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@baf096d[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:30,647 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-15 06:55:30,647 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,648 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,648 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,648 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,648 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,648 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,648 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,648 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,648 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:30,648 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,648 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,649 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:30,649 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-15 06:55:30,649 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-15 06:55:30,649 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,650 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,650 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,652 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-15 06:55:30,652 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,652 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-15 06:55:30,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,40475,1676444032771': 2023-02-15 06:55:30,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-15 06:55:30,654 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-15 06:55:30,655 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:30,655 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:30,655 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-15 06:55:30,655 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:30,656 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:30,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:30,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,656 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,656 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,657 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:30,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,657 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,657 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:30,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,659 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:30,659 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-15 06:55:30,659 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,664 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,664 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,664 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,665 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:30,665 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,665 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-15 06:55:30,665 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:30,664 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,665 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,665 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:30,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,664 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:30,665 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:30,665 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:30,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:30,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,665 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:30,665 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,665 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,665 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:30,666 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,666 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:30,666 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:30,666 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,666 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-15 06:55:30,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:30,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:30,765 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:30,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:31,024 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-15 06:55:31,025 DEBUG [Listener at localhost/37669] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:31,025 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,026 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,027 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,027 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,027 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:31,027 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:31,027 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:31,276 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,276 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:31,276 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:31,276 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:31,276 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-15 06:55:31,276 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:31,276 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:31,276 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:31,276 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:31,277 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:31,278 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-15 06:55:31,279 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:31,279 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:31,279 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:31,279 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:31,279 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:31,280 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:31,280 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:31,280 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:31,280 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:31,280 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:31,280 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:31,280 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:31,281 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:31,281 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:31,281 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:31,281 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:31,281 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:31,281 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:31,281 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:31,282 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:31,282 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:31,282 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:31,282 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:31,282 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:31,282 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:31,282 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,283 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,283 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:31,283 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,283 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,283 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:31,283 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:31,284 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:31,284 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:31,284 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,284 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:31,284 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,284 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,284 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,284 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-15 06:55:31,285 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:31,285 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:31,287 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444161287 2023-02-15 06:55:31,288 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,288 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:31,291 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:31,291 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:31,291 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:31,291 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444191291 2023-02-15 06:55:31,291 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444251291 2023-02-15 06:55:31,291 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:55:31,292 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:31,293 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:31,293 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:31,293 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:31,542 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,542 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:31,542 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:31,542 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:31,542 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:55:31,542 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:31,542 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:31,542 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:31,542 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:31,543 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:31,544 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-15 06:55:31,545 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:31,545 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:31,545 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:31,545 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:31,545 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:31,545 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:31,546 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:31,546 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:31,546 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:31,546 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:31,546 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:31,546 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:31,546 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:31,547 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:31,547 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:31,547 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:31,547 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:31,547 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:31,547 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:31,547 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:31,547 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:31,548 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:31,548 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:31,548 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:31,548 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:31,548 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,548 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,549 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:31,549 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,549 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,549 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:31,549 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:31,549 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:31,549 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:31,549 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,550 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:31,550 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,550 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,550 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,550 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-15 06:55:31,550 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:31,551 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:31,551 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444161551 2023-02-15 06:55:31,553 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,553 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:31,555 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:31,556 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:31,556 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:31,556 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444191556 2023-02-15 06:55:31,556 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444251556 2023-02-15 06:55:31,556 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:55:31,556 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:31,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN}] 2023-02-15 06:55:31,558 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:31,558 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:31,558 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:31,806 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,807 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:31,807 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:31,807 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:31,807 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:55:31,807 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:31,807 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:31,807 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:31,807 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:31,808 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:31,809 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-15 06:55:31,809 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:31,809 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:31,809 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:31,810 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:31,810 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:31,810 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:31,810 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:31,810 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:31,810 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:31,810 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:31,811 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:31,811 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:31,811 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:31,811 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:31,811 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:31,811 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:31,811 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:31,812 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:31,812 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:31,812 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:31,812 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:31,812 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:31,812 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:31,812 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:31,812 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:31,813 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,813 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,813 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:31,813 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,813 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:31,814 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:31,814 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:31,814 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:31,814 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:31,814 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,814 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:31,814 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,814 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,814 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,814 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-15 06:55:31,815 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:31,815 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:31,816 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444161816 2023-02-15 06:55:31,818 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:31,818 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:31,820 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:31,820 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:31,821 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:31,821 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444191821 2023-02-15 06:55:31,821 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444251821 2023-02-15 06:55:31,821 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:55:31,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN}] 2023-02-15 06:55:31,821 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:31,823 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN 2023-02-15 06:55:31,824 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=d29d87088bdb1ecf07d47a3df3023ac0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:31,824 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444131824"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444131824"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444131824"}]},"ts":"1676444131824"} 2023-02-15 06:55:31,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:55:31,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:31,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d29d87088bdb1ecf07d47a3df3023ac0, disabling compactions & flushes 2023-02-15 06:55:31,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:31,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:31,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. after waiting 0 ms 2023-02-15 06:55:31,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:31,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing d29d87088bdb1ecf07d47a3df3023ac0 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-15 06:55:31,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/76c39d4812084894a71f543e671b2f9b 2023-02-15 06:55:32,002 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/9b0c0bd897524cf68d1b83ee61c99f9e 2023-02-15 06:55:32,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf1/76c39d4812084894a71f543e671b2f9b as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b 2023-02-15 06:55:32,010 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:55:32,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/.tmp/cf2/9b0c0bd897524cf68d1b83ee61c99f9e as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e 2023-02-15 06:55:32,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:55:32,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d29d87088bdb1ecf07d47a3df3023ac0 in 37ms, sequenceid=82, compaction requested=true 2023-02-15 06:55:32,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-15 06:55:32,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:32,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. 2023-02-15 06:55:32,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d29d87088bdb1ecf07d47a3df3023ac0: 2023-02-15 06:55:32,022 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,023 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=d29d87088bdb1ecf07d47a3df3023ac0, regionState=CLOSED 2023-02-15 06:55:32,023 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444132023"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444132023"}]},"ts":"1676444132023"} 2023-02-15 06:55:32,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-15 06:55:32,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 in 198 msec 2023-02-15 06:55:32,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-15 06:55:32,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN in 205 msec 2023-02-15 06:55:32,029 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:32,029 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,029 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:32,029 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:32,071 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,071 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:32,071 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:32,071 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:32,071 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:55:32,072 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:32,072 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:32,072 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:32,072 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:32,072 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:32,073 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-15 06:55:32,074 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:55:32,074 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:55:32,074 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:32,074 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:32,074 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:32,074 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:32,075 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:32,075 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:32,075 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:32,075 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:32,075 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:32,075 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:32,075 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:32,076 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:32,076 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:32,076 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:32,076 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:32,076 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:32,076 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:32,077 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:32,077 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:32,077 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:32,077 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:32,077 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:32,077 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:32,077 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:32,077 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:32,078 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:32,078 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:32,078 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:32,078 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:32,079 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:32,079 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:32,079 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:32,079 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:32,080 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:32,080 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,080 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN 2023-02-15 06:55:32,080 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:32,080 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:32,080 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,081 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,081 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,081 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:32,081 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:32,081 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-15 06:55:32,082 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:32,082 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:32,084 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444162084 2023-02-15 06:55:32,084 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:32,086 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,088 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:32,088 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:32,088 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:32,088 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444192088 2023-02-15 06:55:32,088 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444252088 2023-02-15 06:55:32,088 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:55:32,091 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:32,100 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=d29d87088bdb1ecf07d47a3df3023ac0, threads=14 2023-02-15 06:55:32,100 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,100 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,100 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,100 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,100 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,101 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,101 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,101 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,101 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,101 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,101 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,101 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,101 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,101 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,263 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,279 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,292 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,300 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,308 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,316 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,325 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,326 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,328 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,328 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,329 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,329 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,339 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,339 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,339 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region d29d87088bdb1ecf07d47a3df3023ac0 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-15 06:55:32,366 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-15 06:55:32,369 ERROR [BP-145338253-172.31.14.131-1676444028112 heartbeating to localhost/127.0.0.1:41175] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-145338253-172.31.14.131-1676444028112 (Datanode Uuid f7dac7ef-ce9e-4233-96d7-09bfa9824e44) service to localhost/127.0.0.1:41175 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-15 06:55:32,370 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:32,371 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:32,371 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:32,589 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,589 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:32,589 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:32,589 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:32,589 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:55:32,589 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:32,590 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:32,590 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:32,590 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:32,591 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:32,592 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-15 06:55:32,592 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:55:32,592 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:55:32,592 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:32,593 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:32,593 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:32,593 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:32,593 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:32,593 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:32,593 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:32,593 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:32,594 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:32,594 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:32,594 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:32,594 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:32,594 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:32,595 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:32,595 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:32,595 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:32,595 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:32,596 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:32,596 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:32,596 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:32,596 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:32,596 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:32,597 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:32,597 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:32,597 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:32,597 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:32,598 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:32,598 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:32,598 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:32,598 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:32,599 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:32,599 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:32,599 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:32,599 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:32,599 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,600 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN 2023-02-15 06:55:32,600 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:32,600 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:32,600 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,600 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,600 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,600 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:32,600 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:32,601 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-15 06:55:32,601 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:32,602 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:32,606 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444162606 2023-02-15 06:55:32,608 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:32,608 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:32,610 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:32,611 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:32,611 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:32,611 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444192611 2023-02-15 06:55:32,611 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444252611 2023-02-15 06:55:32,611 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-15 06:55:32,619 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=d29d87088bdb1ecf07d47a3df3023ac0, threads=14 2023-02-15 06:55:32,620 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,620 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,624 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,624 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,624 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,624 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,624 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,628 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,628 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,628 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,628 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,628 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,628 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,628 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,764 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,764 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,764 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,764 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,764 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,768 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,780 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,785 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,788 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,792 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,792 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,796 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:32,796 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860 for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:33,119 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e for region: d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:33,119 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region d29d87088bdb1ecf07d47a3df3023ac0 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-15 06:55:33,148 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:33,153 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:55:33,154 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:55:33,154 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:33,154 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:33,154 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:33,361 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,361 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:33,361 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:33,361 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:33,362 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:55:33,362 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:33,362 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:33,362 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:33,362 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:33,363 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:33,364 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-15 06:55:33,364 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:55:33,364 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:55:33,365 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:33,365 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:33,365 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:33,366 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:33,366 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:33,366 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:33,366 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:33,366 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:33,367 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:33,367 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:33,367 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:33,367 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:33,368 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:33,368 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:33,368 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:33,368 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:33,368 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:33,369 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:33,369 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:33,369 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:33,369 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:33,369 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:33,370 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:33,370 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:33,370 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:33,370 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,370 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,371 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:33,371 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,371 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,371 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:33,372 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:33,372 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:33,372 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:33,372 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,372 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN 2023-02-15 06:55:33,372 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:33,372 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:33,373 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,373 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,373 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,373 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:33,373 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:33,373 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-15 06:55:33,374 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:33,374 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:33,374 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444163374 2023-02-15 06:55:33,376 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:33,376 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,378 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:33,378 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:33,378 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:33,378 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444193378 2023-02-15 06:55:33,378 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444253378 2023-02-15 06:55:33,378 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-15 06:55:33,381 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:33,382 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:33,383 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:33,383 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:33,629 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,629 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:33,629 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:33,629 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:33,629 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:55:33,629 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:33,629 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:33,629 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:33,629 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:33,630 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:33,631 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-15 06:55:33,631 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:55:33,631 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:55:33,631 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:33,632 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:33,632 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:33,632 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:33,632 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:33,632 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:33,632 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:33,632 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:33,632 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:33,633 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:33,633 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:33,633 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:33,633 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:33,633 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:33,633 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:33,633 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:33,633 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:33,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:33,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:33,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:33,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:33,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:33,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:33,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:33,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:33,634 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,635 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,635 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:33,635 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,635 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,635 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:33,635 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:33,635 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:33,636 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:33,636 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,636 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN 2023-02-15 06:55:33,636 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:33,636 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:33,636 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,636 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,636 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,636 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:33,636 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:33,636 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-15 06:55:33,637 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:33,637 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:33,638 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444163638 2023-02-15 06:55:33,639 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,639 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:33,641 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:33,642 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:33,642 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:33,642 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444193642 2023-02-15 06:55:33,642 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444253642 2023-02-15 06:55:33,642 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-15 06:55:33,642 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:33,644 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444133644"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676444133644"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676444133644"}]},"ts":"1676444133644"} 2023-02-15 06:55:33,644 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444133644"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444133644"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444133644"}]},"ts":"1676444133644"} 2023-02-15 06:55:33,644 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444133644"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444133644"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444133644"}]},"ts":"1676444133644"} 2023-02-15 06:55:33,646 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:33,646 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:33,646 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:33,892 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,892 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:33,892 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:33,892 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:33,893 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:55:33,893 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:33,893 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:33,893 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:33,893 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:33,894 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:33,895 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-15 06:55:33,895 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:55:33,895 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:55:33,895 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:33,895 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:33,895 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:33,895 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:33,896 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:33,896 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:33,896 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:33,896 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:33,896 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:33,896 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:33,896 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:33,897 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:33,897 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:33,897 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:33,897 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:33,897 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:33,898 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:33,898 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:33,898 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:33,898 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:33,898 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:33,898 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:33,898 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:33,899 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:33,899 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:33,899 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,899 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,899 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:33,900 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,900 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:33,900 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:33,900 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:33,900 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:33,900 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:33,900 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,900 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN 2023-02-15 06:55:33,901 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:33,901 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:33,901 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,901 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,901 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,901 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:33,901 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:33,901 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-15 06:55:33,902 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:33,902 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:33,902 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444163902 2023-02-15 06:55:33,904 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:33,904 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,907 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:33,908 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:33,908 INFO [Listener at localhost/37669] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,40475,1676444032771, table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:33,908 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f35f92b523aab47e93cddd1afad57cbd, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-15 06:55:33,908 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=5fd89f6ba9b04e792f72dadc8f49d434, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-15 06:55:33,908 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:33,908 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444193908 2023-02-15 06:55:33,908 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444253908 2023-02-15 06:55:33,908 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:55:33,909 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676444133908"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676444133908"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676444133908"}]},"ts":"1676444133908"} 2023-02-15 06:55:33,909 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444133908"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444133908"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444133908"}]},"ts":"1676444133908"} 2023-02-15 06:55:33,909 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444133908"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444133908"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444133908"}]},"ts":"1676444133908"} 2023-02-15 06:55:33,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:33,913 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:33,913 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:33,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:33,913 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:34,158 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,159 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:34,159 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:34,159 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:34,159 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:55:34,159 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:34,159 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:34,159 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:34,159 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:34,160 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:34,161 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-15 06:55:34,161 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:55:34,161 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:55:34,161 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:34,162 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:34,162 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:34,162 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:34,162 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:34,162 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:34,162 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:34,162 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:34,163 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:34,163 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:34,163 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:34,163 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:34,163 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:34,163 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:34,163 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:34,164 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:34,164 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:34,164 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:34,164 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:34,164 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:34,164 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:34,164 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:34,164 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:34,164 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:34,165 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:34,165 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,165 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,165 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:34,165 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,165 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,165 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:34,165 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:34,165 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:34,166 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:34,166 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,166 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN 2023-02-15 06:55:34,166 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:34,166 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:34,166 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,166 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,166 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,166 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:34,166 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:34,167 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-15 06:55:34,167 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:34,168 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:34,168 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444164168 2023-02-15 06:55:34,170 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,170 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:34,172 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:34,172 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:34,173 INFO [Listener at localhost/37669] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,40475,1676444032771, table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:34,173 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f35f92b523aab47e93cddd1afad57cbd, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-15 06:55:34,173 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=5fd89f6ba9b04e792f72dadc8f49d434, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-15 06:55:34,173 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:34,173 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444194173 2023-02-15 06:55:34,173 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444254173 2023-02-15 06:55:34,173 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-15 06:55:34,173 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:34,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f35f92b523aab47e93cddd1afad57cbd, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fd89f6ba9b04e792f72dadc8f49d434, ASSIGN}] 2023-02-15 06:55:34,175 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:34,175 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:34,175 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:34,381 ERROR [BP-145338253-172.31.14.131-1676444028112 heartbeating to localhost/127.0.0.1:41175] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-145338253-172.31.14.131-1676444028112 (Datanode Uuid 36321d6c-46ca-442e-92ed-6bcb61ed715a) service to localhost/127.0.0.1:41175 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-15 06:55:34,423 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,423 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:34,423 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:34,424 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:34,424 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:55:34,424 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:34,424 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:34,424 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:34,424 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:34,425 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:34,425 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-15 06:55:34,426 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:55:34,426 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:55:34,426 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:34,426 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:34,426 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:34,426 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:34,427 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:34,427 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:34,427 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:34,427 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:34,427 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:34,427 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:34,427 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:34,427 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:34,428 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:34,428 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:34,428 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:34,428 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:34,428 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:34,428 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:34,428 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:34,428 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:34,429 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:34,429 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:34,429 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:34,429 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:34,429 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:34,429 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,429 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,429 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:34,430 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,430 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,430 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:34,430 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:34,430 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:34,430 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:34,430 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,431 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN 2023-02-15 06:55:34,431 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:34,431 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:34,431 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,431 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,431 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,431 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:34,431 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:34,431 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-15 06:55:34,432 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:34,432 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:34,432 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444164432 2023-02-15 06:55:34,434 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:34,434 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,436 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:34,436 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:34,436 INFO [Listener at localhost/37669] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,40475,1676444032771, table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:34,436 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f35f92b523aab47e93cddd1afad57cbd, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-15 06:55:34,437 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=5fd89f6ba9b04e792f72dadc8f49d434, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-15 06:55:34,437 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:34,437 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444194437 2023-02-15 06:55:34,437 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444254437 2023-02-15 06:55:34,437 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-15 06:55:34,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f35f92b523aab47e93cddd1afad57cbd, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fd89f6ba9b04e792f72dadc8f49d434, ASSIGN}] 2023-02-15 06:55:34,437 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:34,439 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fd89f6ba9b04e792f72dadc8f49d434, ASSIGN 2023-02-15 06:55:34,440 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fd89f6ba9b04e792f72dadc8f49d434, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,40475,1676444032771; forceNewPlan=false, retain=false 2023-02-15 06:55:34,440 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f35f92b523aab47e93cddd1afad57cbd, ASSIGN 2023-02-15 06:55:34,440 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f35f92b523aab47e93cddd1afad57cbd, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,40475,1676444032771; forceNewPlan=false, retain=false 2023-02-15 06:55:34,590 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-15 06:55:34,592 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=5fd89f6ba9b04e792f72dadc8f49d434, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:34,592 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444134592"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444134592"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444134592"}]},"ts":"1676444134592"} 2023-02-15 06:55:34,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 5fd89f6ba9b04e792f72dadc8f49d434, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:55:34,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=f35f92b523aab47e93cddd1afad57cbd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:34,595 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444134595"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444134595"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444134595"}]},"ts":"1676444134595"} 2023-02-15 06:55:34,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure f35f92b523aab47e93cddd1afad57cbd, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:55:34,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:34,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5fd89f6ba9b04e792f72dadc8f49d434, NAME => 'testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.', STARTKEY => '41', ENDKEY => ''} 2023-02-15 06:55:34,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:55:34,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:34,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,750 INFO [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,750 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1 2023-02-15 06:55:34,750 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1 2023-02-15 06:55:34,751 INFO [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fd89f6ba9b04e792f72dadc8f49d434 columnFamilyName cf1 2023-02-15 06:55:34,761 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860-top 2023-02-15 06:55:34,764 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e-top 2023-02-15 06:55:34,767 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb-top 2023-02-15 06:55:34,771 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3-top 2023-02-15 06:55:34,774 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b-top 2023-02-15 06:55:34,777 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058-top 2023-02-15 06:55:34,780 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291-top 2023-02-15 06:55:34,780 INFO [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(310): Store=5fd89f6ba9b04e792f72dadc8f49d434/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:34,780 INFO [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,781 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2 2023-02-15 06:55:34,781 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2 2023-02-15 06:55:34,781 INFO [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5fd89f6ba9b04e792f72dadc8f49d434 columnFamilyName cf2 2023-02-15 06:55:34,792 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf-top 2023-02-15 06:55:34,795 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd-top 2023-02-15 06:55:34,798 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148-top 2023-02-15 06:55:34,801 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e-top 2023-02-15 06:55:34,804 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e-top 2023-02-15 06:55:34,813 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b-top 2023-02-15 06:55:34,816 DEBUG [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b-top 2023-02-15 06:55:34,817 INFO [StoreOpener-5fd89f6ba9b04e792f72dadc8f49d434-1] regionserver.HStore(310): Store=5fd89f6ba9b04e792f72dadc8f49d434/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:34,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:55:34,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5fd89f6ba9b04e792f72dadc8f49d434; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66477950, jitterRate=-0.0094013512134552}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:55:34,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5fd89f6ba9b04e792f72dadc8f49d434: 2023-02-15 06:55:34,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434., pid=117, masterSystemTime=1676444134745 2023-02-15 06:55:34,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:34,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:55:34,825 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:55:34,825 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:55:34,825 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:34,825 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:34,826 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1912): 5fd89f6ba9b04e792f72dadc8f49d434/cf1 is initiating minor compaction (all files) 2023-02-15 06:55:34,826 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1912): 5fd89f6ba9b04e792f72dadc8f49d434/cf2 is initiating minor compaction (all files) 2023-02-15 06:55:34,826 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5fd89f6ba9b04e792f72dadc8f49d434/cf1 in testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:34,826 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5fd89f6ba9b04e792f72dadc8f49d434/cf2 in testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:34,826 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e-top] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/.tmp, totalSize=34.9 K 2023-02-15 06:55:34,826 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b-top] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/.tmp, totalSize=34.9 K 2023-02-15 06:55:34,827 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676444129685 2023-02-15 06:55:34,827 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676444129685 2023-02-15 06:55:34,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:34,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:34,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:34,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f35f92b523aab47e93cddd1afad57cbd, NAME => 'testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.', STARTKEY => '', ENDKEY => '41'} 2023-02-15 06:55:34,828 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676444129879 2023-02-15 06:55:34,828 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676444129879 2023-02-15 06:55:34,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:55:34,828 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=5fd89f6ba9b04e792f72dadc8f49d434, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:34,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:34,828 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444134828"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444134828"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444134828"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444134828"}]},"ts":"1676444134828"} 2023-02-15 06:55:34,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:34,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:34,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:34,828 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676444130065 2023-02-15 06:55:34,829 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676444130065 2023-02-15 06:55:34,829 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676444130244 2023-02-15 06:55:34,829 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676444130244 2023-02-15 06:55:34,830 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676444130415 2023-02-15 06:55:34,830 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676444130415 2023-02-15 06:55:34,830 INFO [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:34,830 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676444130591 2023-02-15 06:55:34,831 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676444130591 2023-02-15 06:55:34,831 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676444130767 2023-02-15 06:55:34,831 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676444130767 2023-02-15 06:55:34,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-15 06:55:34,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 5fd89f6ba9b04e792f72dadc8f49d434, server=jenkins-hbase4.apache.org,40475,1676444032771 in 237 msec 2023-02-15 06:55:34,833 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1 2023-02-15 06:55:34,833 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1 2023-02-15 06:55:34,833 INFO [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f35f92b523aab47e93cddd1afad57cbd columnFamilyName cf1 2023-02-15 06:55:34,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fd89f6ba9b04e792f72dadc8f49d434, ASSIGN in 396 msec 2023-02-15 06:55:34,846 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] throttle.PressureAwareThroughputController(145): 5fd89f6ba9b04e792f72dadc8f49d434#cf2#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:55:34,847 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5fd89f6ba9b04e792f72dadc8f49d434#cf1#compaction#108 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:55:34,849 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860-bottom 2023-02-15 06:55:34,852 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e-bottom 2023-02-15 06:55:34,855 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb-bottom 2023-02-15 06:55:34,858 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3-bottom 2023-02-15 06:55:34,864 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b-bottom 2023-02-15 06:55:34,879 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058-bottom 2023-02-15 06:55:34,881 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/.tmp/cf1/0911fc47dcc54af8821550252c8a5c33 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/0911fc47dcc54af8821550252c8a5c33 2023-02-15 06:55:34,883 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291-bottom 2023-02-15 06:55:34,883 INFO [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(310): Store=f35f92b523aab47e93cddd1afad57cbd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:34,883 INFO [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:34,884 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2 2023-02-15 06:55:34,884 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2 2023-02-15 06:55:34,884 INFO [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f35f92b523aab47e93cddd1afad57cbd columnFamilyName cf2 2023-02-15 06:55:34,885 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5fd89f6ba9b04e792f72dadc8f49d434/cf1 of 5fd89f6ba9b04e792f72dadc8f49d434 into 0911fc47dcc54af8821550252c8a5c33(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:55:34,885 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5fd89f6ba9b04e792f72dadc8f49d434: 2023-02-15 06:55:34,885 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434., storeName=5fd89f6ba9b04e792f72dadc8f49d434/cf1, priority=9, startTime=1676444134824; duration=0sec 2023-02-15 06:55:34,885 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:34,897 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf-bottom 2023-02-15 06:55:34,900 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd-bottom 2023-02-15 06:55:34,903 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148-bottom 2023-02-15 06:55:34,906 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e-bottom 2023-02-15 06:55:34,909 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e-bottom 2023-02-15 06:55:34,912 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b-bottom 2023-02-15 06:55:34,915 DEBUG [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b-bottom 2023-02-15 06:55:34,915 INFO [StoreOpener-f35f92b523aab47e93cddd1afad57cbd-1] regionserver.HStore(310): Store=f35f92b523aab47e93cddd1afad57cbd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:34,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:34,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:34,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:55:34,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:34,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f35f92b523aab47e93cddd1afad57cbd; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63514240, jitterRate=-0.05356407165527344}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:55:34,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f35f92b523aab47e93cddd1afad57cbd: 2023-02-15 06:55:34,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd., pid=118, masterSystemTime=1676444134745 2023-02-15 06:55:34,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:34,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:55:34,922 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:55:34,922 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:34,922 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1912): f35f92b523aab47e93cddd1afad57cbd/cf1 is initiating minor compaction (all files) 2023-02-15 06:55:34,922 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f35f92b523aab47e93cddd1afad57cbd/cf1 in testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:34,923 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b-bottom] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/.tmp, totalSize=34.9 K 2023-02-15 06:55:34,923 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444129685 2023-02-15 06:55:34,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:34,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:34,924 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444129879 2023-02-15 06:55:34,924 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=f35f92b523aab47e93cddd1afad57cbd, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:34,924 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444134924"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444134924"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444134924"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444134924"}]},"ts":"1676444134924"} 2023-02-15 06:55:34,924 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444130065 2023-02-15 06:55:34,925 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444130244 2023-02-15 06:55:34,925 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444130415 2023-02-15 06:55:34,925 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444130591 2023-02-15 06:55:34,926 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] compactions.Compactor(207): Compacting 76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444130767 2023-02-15 06:55:34,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-15 06:55:34,927 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure f35f92b523aab47e93cddd1afad57cbd, server=jenkins-hbase4.apache.org,40475,1676444032771 in 329 msec 2023-02-15 06:55:34,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-15 06:55:34,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f35f92b523aab47e93cddd1afad57cbd, ASSIGN in 490 msec 2023-02-15 06:55:34,928 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-15 06:55:34,928 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:34,928 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-15 06:55:34,934 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] throttle.PressureAwareThroughputController(145): f35f92b523aab47e93cddd1afad57cbd#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:55:34,937 INFO [Listener at localhost/37669] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,937 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-15 06:55:34,937 INFO [Listener at localhost/37669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:34,937 INFO [Listener at localhost/37669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:34,938 ERROR [Listener at localhost/37669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-15 06:55:34,938 INFO [Listener at localhost/37669] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-15 06:55:34,938 INFO [Listener at localhost/37669] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-15 06:55:34,938 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-15 06:55:34,938 INFO [Listener at localhost/37669] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-15 06:55:34,939 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-15 06:55:34,941 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-15 06:55:34,941 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-15 06:55:34,941 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-15 06:55:34,941 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-02-15 06:55:34,941 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-15 06:55:34,941 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-02-15 06:55:34,941 DEBUG [Listener at localhost/37669] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-15 06:55:34,942 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-15 06:55:34,942 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-15 06:55:34,942 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-15 06:55:34,942 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-15 06:55:34,943 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:34,943 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=e93940bab783f139204747313e792776, daughterA=c2a14c04f55c99754ea9727c4d88d1d8, daughterB=a670097e89a3ff97d31f6ef5bcbe626a 2023-02-15 06:55:34,943 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:34,943 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-15 06:55:34,944 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:34,944 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=4593e78b81534378124b8d0b83524921, daughterA=9163391b02da2883a80eb51ae90b7f1c, daughterB=b14b2480de7d04e57425d9cf4365200f 2023-02-15 06:55:34,944 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:34,944 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-15 06:55:34,944 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:34,945 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=f6039dce9e5bf374c6575ade89f0e2df, daughterA=d7c164e31035871a9d949fd6bf789f4d, daughterB=876df0ac8f4998d11d37e33e51b4f60a 2023-02-15 06:55:34,945 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=f6039dce9e5bf374c6575ade89f0e2df, ASSIGN 2023-02-15 06:55:34,945 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:34,945 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-15 06:55:34,946 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:34,946 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=8938ff632984211f5fe6b773e2e60032, daughterA=38f136c511f939e3641e736608f6d72a, daughterB=83267c2682581a47fe295685701d02ca 2023-02-15 06:55:34,946 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:34,946 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-15 06:55:34,946 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:34,947 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=a93cfa0b8140cbc18c2cd8f565cfd003, daughterA=234d84c9a604cb1d5f1618efa32bcb73, daughterB=e51419864fae0902e5f91adf742ee434 2023-02-15 06:55:34,947 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:34,947 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-15 06:55:34,947 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,947 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,948 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,35089,1676444032716, table=testRollbackForSplitTableRegionWithReplica, region=dd4f5e76327a2bdd0e2eb11acd24927a is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=c6d354efa00d71a16486322a96778320, daughterA=2d8ad000e428af8f188a17cb81bd030a, daughterB=fae343cb5c986cdeed9759ea39e40488 2023-02-15 06:55:34,948 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,948 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-15 06:55:34,948 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:34,948 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:34,949 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-15 06:55:34,949 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:34,950 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,950 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN 2023-02-15 06:55:34,950 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:34,950 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f35f92b523aab47e93cddd1afad57cbd, ASSIGN 2023-02-15 06:55:34,951 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fd89f6ba9b04e792f72dadc8f49d434, ASSIGN 2023-02-15 06:55:34,951 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 5fd89f6ba9b04e792f72dadc8f49d434, server=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:34,951 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure f35f92b523aab47e93cddd1afad57cbd, server=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:34,951 DEBUG [Listener at localhost/37669] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 held the lock before restarting, call acquireLock to restore it. 2023-02-15 06:55:34,951 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,951 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,951 INFO [Listener at localhost/37669] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,951 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:34,951 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d29d87088bdb1ecf07d47a3df3023ac0, server=jenkins-hbase4.apache.org,40475,1676444032771 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:34,951 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f35f92b523aab47e93cddd1afad57cbd, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:34,952 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fd89f6ba9b04e792f72dadc8f49d434, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:34,952 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 5fd89f6ba9b04e792f72dadc8f49d434, server=jenkins-hbase4.apache.org,40475,1676444032771 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:34,952 DEBUG [Listener at localhost/37669] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure f35f92b523aab47e93cddd1afad57cbd, server=jenkins-hbase4.apache.org,40475,1676444032771 didn't hold the lock before restarting, skip acquiring lock. 2023-02-15 06:55:34,952 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-15 06:55:34,953 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-15 06:55:34,953 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818, table=hbase:meta, region=1588230740 2023-02-15 06:55:34,953 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676444164953 2023-02-15 06:55:34,953 DEBUG [Listener at localhost/37669] assignment.AssignmentManager(1519): Joining cluster... 2023-02-15 06:55:34,955 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:34,959 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=03b216dbc5303ada2e264ea03896de42, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,35089,1676444032716, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716, openSeqNum=2 2023-02-15 06:55:34,959 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=d29d87088bdb1ecf07d47a3df3023ac0, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=2 2023-02-15 06:55:34,959 INFO [Listener at localhost/37669] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,40475,1676444032771, table=testRecoveryAndDoubleExecution, region=d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:34,959 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=f35f92b523aab47e93cddd1afad57cbd, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=86 2023-02-15 06:55:34,959 DEBUG [Listener at localhost/37669] assignment.RegionStateStore(147): Load hbase:meta entry region=5fd89f6ba9b04e792f72dadc8f49d434, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40475,1676444032771, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771, openSeqNum=86 2023-02-15 06:55:34,960 INFO [Listener at localhost/37669] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-15 06:55:34,960 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676444194960 2023-02-15 06:55:34,960 INFO [Listener at localhost/37669] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676444254960 2023-02-15 06:55:34,960 INFO [Listener at localhost/37669] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-15 06:55:34,960 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-15 06:55:34,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d29d87088bdb1ecf07d47a3df3023ac0, daughterA=f35f92b523aab47e93cddd1afad57cbd, daughterB=5fd89f6ba9b04e792f72dadc8f49d434 in 3.9350 sec 2023-02-15 06:55:35,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40475] ipc.CallRunner(144): callId: 858 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:59810 deadline: 1676444195211, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0. is not online on jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:35,275 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/.tmp/cf2/cc4bdcca548a44c285fe327f95cd117f as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/cc4bdcca548a44c285fe327f95cd117f 2023-02-15 06:55:35,279 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5fd89f6ba9b04e792f72dadc8f49d434/cf2 of 5fd89f6ba9b04e792f72dadc8f49d434 into cc4bdcca548a44c285fe327f95cd117f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:55:35,279 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5fd89f6ba9b04e792f72dadc8f49d434: 2023-02-15 06:55:35,279 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434., storeName=5fd89f6ba9b04e792f72dadc8f49d434/cf2, priority=9, startTime=1676444134825; duration=0sec 2023-02-15 06:55:35,279 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:55:35,279 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:55:35,279 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:35,279 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1912): f35f92b523aab47e93cddd1afad57cbd/cf2 is initiating minor compaction (all files) 2023-02-15 06:55:35,279 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f35f92b523aab47e93cddd1afad57cbd/cf2 in testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:35,280 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e-bottom] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/.tmp, totalSize=34.9 K 2023-02-15 06:55:35,280 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444129685 2023-02-15 06:55:35,281 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444129879 2023-02-15 06:55:35,281 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444130065 2023-02-15 06:55:35,281 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444130244 2023-02-15 06:55:35,282 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444130415 2023-02-15 06:55:35,282 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444130591 2023-02-15 06:55:35,283 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] compactions.Compactor(207): Compacting 9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444130767 2023-02-15 06:55:35,292 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] throttle.PressureAwareThroughputController(145): f35f92b523aab47e93cddd1afad57cbd#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:55:35,319 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-15 06:55:35,327 INFO [Listener at localhost/37669] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-15 06:55:35,328 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-15 06:55:35,351 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/.tmp/cf1/10aab537c9a74a6d92178ee5ba19ec19 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/10aab537c9a74a6d92178ee5ba19ec19 2023-02-15 06:55:35,353 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:55:35,354 INFO [Listener at localhost/37669] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-15 06:55:35,355 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-15 06:55:35,355 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f35f92b523aab47e93cddd1afad57cbd/cf1 of f35f92b523aab47e93cddd1afad57cbd into 10aab537c9a74a6d92178ee5ba19ec19(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:55:35,355 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f35f92b523aab47e93cddd1afad57cbd: 2023-02-15 06:55:35,355 INFO [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd., storeName=f35f92b523aab47e93cddd1afad57cbd/cf1, priority=9, startTime=1676444134921; duration=0sec 2023-02-15 06:55:35,355 DEBUG [RS:1;jenkins-hbase4:40475-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:35,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:35,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-15 06:55:35,358 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444135358"}]},"ts":"1676444135358"} 2023-02-15 06:55:35,359 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-15 06:55:35,365 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-15 06:55:35,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f35f92b523aab47e93cddd1afad57cbd, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fd89f6ba9b04e792f72dadc8f49d434, UNASSIGN}] 2023-02-15 06:55:35,366 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fd89f6ba9b04e792f72dadc8f49d434, UNASSIGN 2023-02-15 06:55:35,367 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=5fd89f6ba9b04e792f72dadc8f49d434, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:35,367 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444135367"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444135367"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444135367"}]},"ts":"1676444135367"} 2023-02-15 06:55:35,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 5fd89f6ba9b04e792f72dadc8f49d434, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:55:35,370 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f35f92b523aab47e93cddd1afad57cbd, UNASSIGN 2023-02-15 06:55:35,370 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=f35f92b523aab47e93cddd1afad57cbd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:35,370 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444135370"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444135370"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444135370"}]},"ts":"1676444135370"} 2023-02-15 06:55:35,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure f35f92b523aab47e93cddd1afad57cbd, server=jenkins-hbase4.apache.org,40475,1676444032771}] 2023-02-15 06:55:35,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-15 06:55:35,520 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:35,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f35f92b523aab47e93cddd1afad57cbd, disabling compactions & flushes 2023-02-15 06:55:35,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:35,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-15 06:55:35,702 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/.tmp/cf2/28794b6abde440eeb297f862c228987c as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/28794b6abde440eeb297f862c228987c 2023-02-15 06:55:35,706 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f35f92b523aab47e93cddd1afad57cbd/cf2 of f35f92b523aab47e93cddd1afad57cbd into 28794b6abde440eeb297f862c228987c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:55:35,706 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f35f92b523aab47e93cddd1afad57cbd: 2023-02-15 06:55:35,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:35,706 INFO [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd., storeName=f35f92b523aab47e93cddd1afad57cbd/cf2, priority=9, startTime=1676444134922; duration=0sec 2023-02-15 06:55:35,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:35,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. after waiting 0 ms 2023-02-15 06:55:35,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:35,706 DEBUG [RS:1;jenkins-hbase4:40475-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:35,707 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b-bottom] to archive 2023-02-15 06:55:35,707 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:55:35,708 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,709 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,710 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,711 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,712 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,713 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,714 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,716 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e-bottom] to archive 2023-02-15 06:55:35,716 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:55:35,718 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,719 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,720 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,720 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,721 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,722 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,723 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-15 06:55:35,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:35,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd. 2023-02-15 06:55:35,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f35f92b523aab47e93cddd1afad57cbd: 2023-02-15 06:55:35,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:35,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:35,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5fd89f6ba9b04e792f72dadc8f49d434, disabling compactions & flushes 2023-02-15 06:55:35,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:35,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:35,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. after waiting 0 ms 2023-02-15 06:55:35,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:35,730 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=f35f92b523aab47e93cddd1afad57cbd, regionState=CLOSED 2023-02-15 06:55:35,730 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444135730"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444135730"}]},"ts":"1676444135730"} 2023-02-15 06:55:35,731 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b-top] to archive 2023-02-15 06:55:35,732 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:55:35,734 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,735 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-15 06:55:35,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure f35f92b523aab47e93cddd1afad57cbd, server=jenkins-hbase4.apache.org,40475,1676444032771 in 360 msec 2023-02-15 06:55:35,736 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,737 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,738 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,740 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=f35f92b523aab47e93cddd1afad57cbd, UNASSIGN in 370 msec 2023-02-15 06:55:35,742 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,744 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e-top] to archive 2023-02-15 06:55:35,745 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:55:35,746 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,747 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,748 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,749 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,750 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,751 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,752 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-15 06:55:35,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:35,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434. 2023-02-15 06:55:35,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5fd89f6ba9b04e792f72dadc8f49d434: 2023-02-15 06:55:35,757 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:35,758 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=5fd89f6ba9b04e792f72dadc8f49d434, regionState=CLOSED 2023-02-15 06:55:35,758 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676444135758"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444135758"}]},"ts":"1676444135758"} 2023-02-15 06:55:35,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-15 06:55:35,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 5fd89f6ba9b04e792f72dadc8f49d434, server=jenkins-hbase4.apache.org,40475,1676444032771 in 391 msec 2023-02-15 06:55:35,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-02-15 06:55:35,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=5fd89f6ba9b04e792f72dadc8f49d434, UNASSIGN in 395 msec 2023-02-15 06:55:35,762 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444135762"}]},"ts":"1676444135762"} 2023-02-15 06:55:35,763 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-15 06:55:35,765 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-15 06:55:35,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 410 msec 2023-02-15 06:55:35,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-15 06:55:35,962 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-15 06:55:35,963 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-15 06:55:35,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:35,965 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:35,966 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:35,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-15 06:55:35,970 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,970 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:35,970 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:35,972 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/recovered.edits] 2023-02-15 06:55:35,972 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/recovered.edits] 2023-02-15 06:55:35,972 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/recovered.edits] 2023-02-15 06:55:35,975 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/10aab537c9a74a6d92178ee5ba19ec19 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf1/10aab537c9a74a6d92178ee5ba19ec19 2023-02-15 06:55:35,976 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/.regioninfo to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/.regioninfo 2023-02-15 06:55:35,977 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/06fcad3fc84d487ca90f6f5ec9c98860 2023-02-15 06:55:35,978 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/15c746103b024fb587cc8160cac4fa9e 2023-02-15 06:55:35,978 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/28794b6abde440eeb297f862c228987c to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/cf2/28794b6abde440eeb297f862c228987c 2023-02-15 06:55:35,979 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,979 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/5075082840b04a81acd1f17d868d45bb 2023-02-15 06:55:35,980 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,984 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/645b51fa33e848b29c29942ee2c36ac3 2023-02-15 06:55:35,984 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/.regioninfo to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/.regioninfo 2023-02-15 06:55:35,985 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,985 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/76c39d4812084894a71f543e671b2f9b 2023-02-15 06:55:35,987 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,987 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/eca3f849060c4dc2bdf9df512c78d058 2023-02-15 06:55:35,987 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/06fcad3fc84d487ca90f6f5ec9c98860.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,988 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,988 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291 2023-02-15 06:55:35,989 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/15c746103b024fb587cc8160cac4fa9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,989 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,990 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/5075082840b04a81acd1f17d868d45bb.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,991 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/645b51fa33e848b29c29942ee2c36ac3.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,991 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/09cdca57f96541bfb09f40e41376dedf 2023-02-15 06:55:35,991 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,993 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/76c39d4812084894a71f543e671b2f9b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,993 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ac3e26234464b63a22bd50772d242dd 2023-02-15 06:55:35,994 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/eca3f849060c4dc2bdf9df512c78d058.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,994 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/0ee22f032c2e41f486e4a61998c71148 2023-02-15 06:55:35,995 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,996 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/7eec12b2504446f8b0d0bb81d497dd9e 2023-02-15 06:55:35,996 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf1/ff9cc6d8ab1b4252acff9a16c5e4b291.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,996 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:35,997 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/9b0c0bd897524cf68d1b83ee61c99f9e 2023-02-15 06:55:36,000 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,001 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/deae548396ce40b6987ba172e3de406b 2023-02-15 06:55:36,002 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/09cdca57f96541bfb09f40e41376dedf.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,002 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,003 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/cf2/e2cd97c6613848c9b1b11b7017f2981b 2023-02-15 06:55:36,003 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/0ac3e26234464b63a22bd50772d242dd.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,003 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,004 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/0ee22f032c2e41f486e4a61998c71148.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,004 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,006 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/5fd89f6ba9b04e792f72dadc8f49d434/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,006 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/recovered.edits/85.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0/recovered.edits/85.seqid 2023-02-15 06:55:36,006 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/7eec12b2504446f8b0d0bb81d497dd9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,006 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,007 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/9b0c0bd897524cf68d1b83ee61c99f9e.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,008 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/0911fc47dcc54af8821550252c8a5c33 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf1/0911fc47dcc54af8821550252c8a5c33 2023-02-15 06:55:36,008 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/deae548396ce40b6987ba172e3de406b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,009 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/f35f92b523aab47e93cddd1afad57cbd/cf2/e2cd97c6613848c9b1b11b7017f2981b.d29d87088bdb1ecf07d47a3df3023ac0 2023-02-15 06:55:36,010 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/cc4bdcca548a44c285fe327f95cd117f to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/cf2/cc4bdcca548a44c285fe327f95cd117f 2023-02-15 06:55:36,012 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/recovered.edits/90.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd/recovered.edits/90.seqid 2023-02-15 06:55:36,012 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/f35f92b523aab47e93cddd1afad57cbd 2023-02-15 06:55:36,012 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/recovered.edits/90.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434/recovered.edits/90.seqid 2023-02-15 06:55:36,013 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testRecoveryAndDoubleExecution/5fd89f6ba9b04e792f72dadc8f49d434 2023-02-15 06:55:36,013 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-15 06:55:36,015 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:36,017 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-15 06:55:36,018 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-15 06:55:36,019 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:36,019 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-15 06:55:36,019 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444136019"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:36,019 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444136019"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:36,019 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444136019"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:36,020 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-15 06:55:36,021 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d29d87088bdb1ecf07d47a3df3023ac0, NAME => 'testRecoveryAndDoubleExecution,,1676444129181.d29d87088bdb1ecf07d47a3df3023ac0.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => f35f92b523aab47e93cddd1afad57cbd, NAME => 'testRecoveryAndDoubleExecution,,1676444131025.f35f92b523aab47e93cddd1afad57cbd.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 5fd89f6ba9b04e792f72dadc8f49d434, NAME => 'testRecoveryAndDoubleExecution,41,1676444131025.5fd89f6ba9b04e792f72dadc8f49d434.', STARTKEY => '41', ENDKEY => ''}] 2023-02-15 06:55:36,021 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-15 06:55:36,021 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676444136021"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:36,022 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-15 06:55:36,024 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-15 06:55:36,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 61 msec 2023-02-15 06:55:36,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-15 06:55:36,067 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-15 06:55:36,082 INFO [Listener at localhost/37669] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=376 (was 369) - Thread LEAK? -, OpenFileDescriptor=844 (was 744) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=147 (was 151), ProcessCount=168 (was 168), AvailableMemoryMB=3197 (was 3209) 2023-02-15 06:55:36,095 INFO [Listener at localhost/37669] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376, OpenFileDescriptor=844, MaxFileDescriptor=60000, SystemLoadAverage=147, ProcessCount=168, AvailableMemoryMB=3197 2023-02-15 06:55:36,095 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:55:36,098 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-15 06:55:36,098 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-15 06:55:36,099 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-15 06:55:36,101 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:55:36,101 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:55:36,102 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:36,103 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7 empty. 2023-02-15 06:55:36,103 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:36,103 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-15 06:55:36,512 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-15 06:55:36,513 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8201b09c78dcdd78cf4aad35dd3691a7, NAME => 'testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:55:36,520 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:36,520 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 8201b09c78dcdd78cf4aad35dd3691a7, disabling compactions & flushes 2023-02-15 06:55:36,520 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:36,520 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:36,520 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. after waiting 0 ms 2023-02-15 06:55:36,520 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:36,520 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:36,521 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 8201b09c78dcdd78cf4aad35dd3691a7: 2023-02-15 06:55:36,522 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:55:36,523 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444136523"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444136523"}]},"ts":"1676444136523"} 2023-02-15 06:55:36,524 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:55:36,525 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:55:36,525 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444136525"}]},"ts":"1676444136525"} 2023-02-15 06:55:36,526 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-15 06:55:36,530 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:55:36,531 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:55:36,531 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:55:36,531 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:55:36,531 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:55:36,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8201b09c78dcdd78cf4aad35dd3691a7, ASSIGN}] 2023-02-15 06:55:36,532 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8201b09c78dcdd78cf4aad35dd3691a7, ASSIGN 2023-02-15 06:55:36,533 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8201b09c78dcdd78cf4aad35dd3691a7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,35089,1676444032716; forceNewPlan=false, retain=false 2023-02-15 06:55:36,683 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:55:36,684 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=8201b09c78dcdd78cf4aad35dd3691a7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:36,684 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444136684"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444136684"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444136684"}]},"ts":"1676444136684"} 2023-02-15 06:55:36,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 8201b09c78dcdd78cf4aad35dd3691a7, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:55:36,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:36,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8201b09c78dcdd78cf4aad35dd3691a7, NAME => 'testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:55:36,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:55:36,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:36,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:36,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:36,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:36,844 INFO [StoreOpener-8201b09c78dcdd78cf4aad35dd3691a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:36,845 DEBUG [StoreOpener-8201b09c78dcdd78cf4aad35dd3691a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1 2023-02-15 06:55:36,845 DEBUG [StoreOpener-8201b09c78dcdd78cf4aad35dd3691a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1 2023-02-15 06:55:36,845 INFO [StoreOpener-8201b09c78dcdd78cf4aad35dd3691a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8201b09c78dcdd78cf4aad35dd3691a7 columnFamilyName cf1 2023-02-15 06:55:36,846 INFO [StoreOpener-8201b09c78dcdd78cf4aad35dd3691a7-1] regionserver.HStore(310): Store=8201b09c78dcdd78cf4aad35dd3691a7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:36,846 INFO [StoreOpener-8201b09c78dcdd78cf4aad35dd3691a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:36,847 DEBUG [StoreOpener-8201b09c78dcdd78cf4aad35dd3691a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2 2023-02-15 06:55:36,847 DEBUG [StoreOpener-8201b09c78dcdd78cf4aad35dd3691a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2 2023-02-15 06:55:36,847 INFO [StoreOpener-8201b09c78dcdd78cf4aad35dd3691a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8201b09c78dcdd78cf4aad35dd3691a7 columnFamilyName cf2 2023-02-15 06:55:36,847 INFO [StoreOpener-8201b09c78dcdd78cf4aad35dd3691a7-1] regionserver.HStore(310): Store=8201b09c78dcdd78cf4aad35dd3691a7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:36,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:36,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:36,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:55:36,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:36,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:55:36,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8201b09c78dcdd78cf4aad35dd3691a7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67292868, jitterRate=0.002741873264312744}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:55:36,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8201b09c78dcdd78cf4aad35dd3691a7: 2023-02-15 06:55:36,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7., pid=127, masterSystemTime=1676444136837 2023-02-15 06:55:36,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:36,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:36,855 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=8201b09c78dcdd78cf4aad35dd3691a7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:36,855 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444136855"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444136855"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444136855"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444136855"}]},"ts":"1676444136855"} 2023-02-15 06:55:36,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-15 06:55:36,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 8201b09c78dcdd78cf4aad35dd3691a7, server=jenkins-hbase4.apache.org,35089,1676444032716 in 170 msec 2023-02-15 06:55:36,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-15 06:55:36,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8201b09c78dcdd78cf4aad35dd3691a7, ASSIGN in 326 msec 2023-02-15 06:55:36,859 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:55:36,859 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444136859"}]},"ts":"1676444136859"} 2023-02-15 06:55:36,860 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-15 06:55:36,862 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:55:36,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 763 msec 2023-02-15 06:55:37,107 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:37,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,108 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,108 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,108 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-15 06:55:37,108 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-15 06:55:37,109 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,109 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,110 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,110 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,110 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,110 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,111 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,111 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,112 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:37,113 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,113 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. started... 2023-02-15 06:55:37,113 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:37,113 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,114 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8201b09c78dcdd78cf4aad35dd3691a7 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-15 06:55:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,120 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,120 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,120 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,120 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,120 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,125 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/efea034835904f319f8e790b58bfabb5 2023-02-15 06:55:37,135 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/902cafaccf25480b9f207bae187f7f78 2023-02-15 06:55:37,139 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/efea034835904f319f8e790b58bfabb5 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/efea034835904f319f8e790b58bfabb5 2023-02-15 06:55:37,142 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/efea034835904f319f8e790b58bfabb5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:55:37,142 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/902cafaccf25480b9f207bae187f7f78 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/902cafaccf25480b9f207bae187f7f78 2023-02-15 06:55:37,146 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/902cafaccf25480b9f207bae187f7f78, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:55:37,146 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 8201b09c78dcdd78cf4aad35dd3691a7 in 32ms, sequenceid=6, compaction requested=false 2023-02-15 06:55:37,146 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,147 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8201b09c78dcdd78cf4aad35dd3691a7: 2023-02-15 06:55:37,147 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:37,147 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:37,147 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:37,147 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,147 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,147 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,149 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,149 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,149 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,149 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-15 06:55:37,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3fd83fe4[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:37,151 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-15 06:55:37,151 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,152 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,152 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,152 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,152 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,152 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:37,152 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,152 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,152 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:37,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,153 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,153 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,153 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,153 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,153 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:37,153 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:37,160 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,160 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,160 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,161 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,161 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,161 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,161 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,161 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,161 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,161 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:55:37,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:55:37,164 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-15 06:55:37,164 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:37,164 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:37,164 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,164 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:37,165 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,165 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,165 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,166 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,165 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,165 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,166 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,166 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,166 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,167 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,168 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,168 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,169 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,183 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,183 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,183 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,183 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,183 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,183 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:37,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,183 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,183 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,183 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,184 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:37,183 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,183 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,184 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,184 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:37,184 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,184 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,185 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,185 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,185 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,185 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,185 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,185 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,284 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:37,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:37,296 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:37,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:37,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,298 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,298 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,298 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-15 06:55:37,298 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-15 06:55:37,298 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,298 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,299 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,299 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,299 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,299 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,300 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,300 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:37,300 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,300 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,301 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,301 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,301 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,301 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,301 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,301 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,301 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:37,301 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,302 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,302 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. started... 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,302 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8201b09c78dcdd78cf4aad35dd3691a7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,302 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,303 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,303 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,304 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,304 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,304 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,304 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,312 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/16ba9ee4eb9842c98cafbc2548b293e8 2023-02-15 06:55:37,322 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/1157c02ab7f54d1d9c18db9c4956205d 2023-02-15 06:55:37,325 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/16ba9ee4eb9842c98cafbc2548b293e8 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/16ba9ee4eb9842c98cafbc2548b293e8 2023-02-15 06:55:37,328 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/16ba9ee4eb9842c98cafbc2548b293e8, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:55:37,329 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/1157c02ab7f54d1d9c18db9c4956205d as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/1157c02ab7f54d1d9c18db9c4956205d 2023-02-15 06:55:37,332 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/1157c02ab7f54d1d9c18db9c4956205d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:55:37,333 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8201b09c78dcdd78cf4aad35dd3691a7 in 31ms, sequenceid=19, compaction requested=false 2023-02-15 06:55:37,333 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8201b09c78dcdd78cf4aad35dd3691a7: 2023-02-15 06:55:37,333 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:37,333 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:37,333 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:37,333 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,333 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,333 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,335 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,335 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,335 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,335 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-15 06:55:37,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a468da2[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:37,337 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-15 06:55:37,337 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,339 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,339 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,339 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,339 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,339 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,339 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:37,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,339 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:37,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,339 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,340 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,340 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,340 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,340 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:37,340 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:37,341 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,341 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,341 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,342 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,342 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,342 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,342 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,342 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,342 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,342 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:55:37,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:55:37,344 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-15 06:55:37,345 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:37,345 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:37,345 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,345 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:37,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,346 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,347 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,347 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,347 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,347 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,349 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,349 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,350 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,353 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,353 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,354 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,354 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,353 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,354 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,354 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:37,354 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,354 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:37,354 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,354 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,354 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:37,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,354 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,355 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,355 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,355 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,355 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,355 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,355 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,355 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,355 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,355 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,455 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:37,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:37,466 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:37,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:37,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,468 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,468 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,468 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-15 06:55:37,468 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-15 06:55:37,468 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,468 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,469 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,469 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,469 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,469 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,470 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,470 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:37,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,471 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,471 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,471 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,471 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,471 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,471 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,471 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,471 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,471 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,471 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,472 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,472 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,472 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,472 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,472 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,473 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,473 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,473 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,474 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,474 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,474 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,479 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:37,479 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:37,479 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:37,479 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. started... 2023-02-15 06:55:37,479 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8201b09c78dcdd78cf4aad35dd3691a7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:37,494 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/061351af8a2742ccac1b54dfda902d95 2023-02-15 06:55:37,511 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/a986a6cad34842d394f79f0e7a863a3a 2023-02-15 06:55:37,515 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/061351af8a2742ccac1b54dfda902d95 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/061351af8a2742ccac1b54dfda902d95 2023-02-15 06:55:37,518 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/061351af8a2742ccac1b54dfda902d95, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:55:37,519 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/a986a6cad34842d394f79f0e7a863a3a as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/a986a6cad34842d394f79f0e7a863a3a 2023-02-15 06:55:37,521 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-15 06:55:37,522 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/a986a6cad34842d394f79f0e7a863a3a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:55:37,523 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8201b09c78dcdd78cf4aad35dd3691a7 in 44ms, sequenceid=32, compaction requested=true 2023-02-15 06:55:37,523 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8201b09c78dcdd78cf4aad35dd3691a7: 2023-02-15 06:55:37,523 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:37,523 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:37,523 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:37,523 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,523 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,523 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,526 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,526 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,526 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,526 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-15 06:55:37,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73197109[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:37,528 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-15 06:55:37,528 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,530 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,530 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,530 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,530 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,530 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,530 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:37,530 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,530 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,530 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,530 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:37,530 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,530 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:37,530 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,531 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:37,535 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,535 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,535 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,535 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,537 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,537 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,537 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,537 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,537 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,537 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:55:37,541 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:55:37,541 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-15 06:55:37,541 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:37,541 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:37,541 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,541 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:37,543 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,543 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,543 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,543 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,543 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,543 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,543 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,545 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,545 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,546 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,547 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,548 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,548 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,551 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,551 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,551 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,551 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,551 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,551 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,551 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,551 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,552 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,552 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,552 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,552 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,552 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,552 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,552 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,552 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,552 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:37,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,553 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:37,552 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,553 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:37,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,552 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,553 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,653 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:37,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:37,665 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:37,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:37,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,666 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,666 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,666 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-15 06:55:37,666 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-15 06:55:37,666 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,666 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,668 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,668 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,668 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,668 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:37,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,669 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,670 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:37,670 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,670 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. started... 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,670 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,670 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8201b09c78dcdd78cf4aad35dd3691a7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,673 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,681 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/b5c3b9df2d04404fb1f1a66997515797 2023-02-15 06:55:37,691 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/aa18d11c467941f3aed635e42a1661e9 2023-02-15 06:55:37,694 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/b5c3b9df2d04404fb1f1a66997515797 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/b5c3b9df2d04404fb1f1a66997515797 2023-02-15 06:55:37,698 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/b5c3b9df2d04404fb1f1a66997515797, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:55:37,698 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/aa18d11c467941f3aed635e42a1661e9 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/aa18d11c467941f3aed635e42a1661e9 2023-02-15 06:55:37,702 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/aa18d11c467941f3aed635e42a1661e9, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:55:37,702 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8201b09c78dcdd78cf4aad35dd3691a7 in 32ms, sequenceid=45, compaction requested=true 2023-02-15 06:55:37,703 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8201b09c78dcdd78cf4aad35dd3691a7: 2023-02-15 06:55:37,703 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:37,703 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:37,703 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:37,703 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,703 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,703 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,704 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,704 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,704 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,704 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-15 06:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@eea66[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:37,706 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-15 06:55:37,706 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,707 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,707 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,707 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:37,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,708 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,708 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,708 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,708 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,708 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,708 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,708 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:37,708 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:37,708 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:37,709 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,709 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,709 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,709 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,709 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,709 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,709 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,710 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,709 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,709 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:55:37,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:55:37,712 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-15 06:55:37,712 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:37,712 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:37,712 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,712 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:37,715 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,715 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,715 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,715 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,715 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,715 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,715 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,715 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,716 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,718 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,720 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,721 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,721 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,725 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,725 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,725 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,725 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,725 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,725 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,725 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,725 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,726 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:37,725 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,725 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,726 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,726 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:37,726 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,726 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:37,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,726 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,726 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,726 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,726 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,726 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,727 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,727 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,826 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:37,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:37,838 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:37,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:37,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,840 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,840 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,840 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-15 06:55:37,840 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-15 06:55:37,840 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,840 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,841 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,841 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,842 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,841 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,842 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,842 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,843 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,843 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,843 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,843 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,843 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,843 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,843 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,843 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,843 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:37,843 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,844 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,844 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,844 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:37,844 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,844 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:37,844 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,844 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:37,844 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:37,844 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:37,844 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:37,844 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. started... 2023-02-15 06:55:37,844 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8201b09c78dcdd78cf4aad35dd3691a7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:37,846 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,846 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,846 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,846 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,847 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,847 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,852 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/21f58da3123847dfb5a4e3a99a5665b0 2023-02-15 06:55:37,862 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/47ea04940ff244cab7562c7530b8c4e2 2023-02-15 06:55:37,866 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/21f58da3123847dfb5a4e3a99a5665b0 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/21f58da3123847dfb5a4e3a99a5665b0 2023-02-15 06:55:37,869 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/21f58da3123847dfb5a4e3a99a5665b0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:55:37,869 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/47ea04940ff244cab7562c7530b8c4e2 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/47ea04940ff244cab7562c7530b8c4e2 2023-02-15 06:55:37,872 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/47ea04940ff244cab7562c7530b8c4e2, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:55:37,873 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8201b09c78dcdd78cf4aad35dd3691a7 in 29ms, sequenceid=58, compaction requested=true 2023-02-15 06:55:37,873 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8201b09c78dcdd78cf4aad35dd3691a7: 2023-02-15 06:55:37,873 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:37,873 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:37,873 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:37,873 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,873 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:37,873 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:37,874 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,874 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,875 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,875 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:37,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-15 06:55:37,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ed7d29c[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:37,876 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-15 06:55:37,877 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,877 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,877 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,878 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:37,878 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,878 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:37,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,881 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:37,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:55:37,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:55:37,884 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-15 06:55:37,884 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:37,884 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:37,884 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,884 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:37,886 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,886 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:37,886 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:37,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,887 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,887 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:37,889 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,889 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,889 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,891 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,891 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,891 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,891 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,892 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:37,892 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:37,892 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:37,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:37,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:37,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:37,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,892 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:37,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:37,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:37,893 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:37,893 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:37,893 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,893 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:37,993 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:37,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:38,003 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:38,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:38,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,005 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:38,005 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:38,005 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-15 06:55:38,005 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-15 06:55:38,005 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,005 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,008 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:38,008 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:38,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:38,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:38,008 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,008 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:38,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,008 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,008 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:38,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,009 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-15 06:55:38,009 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:38,009 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:38,009 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:38,010 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:38,010 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. started... 2023-02-15 06:55:38,010 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8201b09c78dcdd78cf4aad35dd3691a7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:38,011 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,012 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,012 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,012 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:38,012 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,012 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:38,018 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/3894842d396d4e2f8f4ef8f823e90f5d 2023-02-15 06:55:38,028 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/ec96646855974ba79554f324f09a1cec 2023-02-15 06:55:38,031 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/3894842d396d4e2f8f4ef8f823e90f5d as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/3894842d396d4e2f8f4ef8f823e90f5d 2023-02-15 06:55:38,034 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/3894842d396d4e2f8f4ef8f823e90f5d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:55:38,035 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/ec96646855974ba79554f324f09a1cec as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/ec96646855974ba79554f324f09a1cec 2023-02-15 06:55:38,038 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/ec96646855974ba79554f324f09a1cec, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:55:38,039 INFO [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8201b09c78dcdd78cf4aad35dd3691a7 in 29ms, sequenceid=71, compaction requested=true 2023-02-15 06:55:38,039 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8201b09c78dcdd78cf4aad35dd3691a7: 2023-02-15 06:55:38,039 DEBUG [rs(jenkins-hbase4.apache.org,35089,1676444032716)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:38,039 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:38,039 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:38,039 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,039 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-15 06:55:38,039 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-15 06:55:38,040 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,040 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:38,041 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,041 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:38,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:38,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:38,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:38,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-15 06:55:38,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40c27b79[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:38,043 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-15 06:55:38,043 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,049 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,049 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,049 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,049 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,049 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:38,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:38,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-15 06:55:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-15 06:55:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:38,049 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:38,051 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,051 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:38,051 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:38,051 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:38,051 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:38,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,051 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:38,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:38,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:38,051 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:38,051 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-15 06:55:38,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:38,051 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:38,051 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-15 06:55:38,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:38,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:38,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:38,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:38,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:38,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:38,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,35089,1676444032716': 2023-02-15 06:55:38,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-15 06:55:38,054 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-15 06:55:38,054 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:38,054 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:38,054 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,054 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:38,055 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:38,056 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,056 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:38,056 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:38,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:38,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:38,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:38,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:38,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:38,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:38,056 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:38,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:38,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:38,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:38,057 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:38,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:38,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:38,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:38,058 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:38,059 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:38,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,059 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:38,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:38,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:38,062 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:38,062 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-15 06:55:38,063 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:38,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:38,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:38,062 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:38,063 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:38,063 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:38,063 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:38,063 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:38,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:38,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:38,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:38,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:38,063 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,063 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:38,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:38,063 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:38,063 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-15 06:55:38,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:38,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:38,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:38,163 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:38,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:38,171 DEBUG [Listener at localhost/37669] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,172 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=8201b09c78dcdd78cf4aad35dd3691a7, daughterA=d44023a1ed274fb3f64074c6a39d6b0c, daughterB=b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,173 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=8201b09c78dcdd78cf4aad35dd3691a7, daughterA=d44023a1ed274fb3f64074c6a39d6b0c, daughterB=b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,173 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=8201b09c78dcdd78cf4aad35dd3691a7, daughterA=d44023a1ed274fb3f64074c6a39d6b0c, daughterB=b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,173 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=8201b09c78dcdd78cf4aad35dd3691a7, daughterA=d44023a1ed274fb3f64074c6a39d6b0c, daughterB=b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8201b09c78dcdd78cf4aad35dd3691a7, UNASSIGN}] 2023-02-15 06:55:38,175 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8201b09c78dcdd78cf4aad35dd3691a7, UNASSIGN 2023-02-15 06:55:38,176 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=8201b09c78dcdd78cf4aad35dd3691a7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,176 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444138176"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444138176"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444138176"}]},"ts":"1676444138176"} 2023-02-15 06:55:38,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 8201b09c78dcdd78cf4aad35dd3691a7, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:55:38,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8201b09c78dcdd78cf4aad35dd3691a7, disabling compactions & flushes 2023-02-15 06:55:38,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:38,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:38,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. after waiting 0 ms 2023-02-15 06:55:38,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:38,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 8201b09c78dcdd78cf4aad35dd3691a7 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-15 06:55:38,340 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/6850ccdb2da741909957ff3395c87635 2023-02-15 06:55:38,353 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/044f180860354ff78bfadaba17ae29bd 2023-02-15 06:55:38,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf1/6850ccdb2da741909957ff3395c87635 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/6850ccdb2da741909957ff3395c87635 2023-02-15 06:55:38,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/6850ccdb2da741909957ff3395c87635, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:55:38,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.tmp/cf2/044f180860354ff78bfadaba17ae29bd as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/044f180860354ff78bfadaba17ae29bd 2023-02-15 06:55:38,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/044f180860354ff78bfadaba17ae29bd, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:55:38,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 8201b09c78dcdd78cf4aad35dd3691a7 in 35ms, sequenceid=82, compaction requested=true 2023-02-15 06:55:38,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-15 06:55:38,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:38,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. 2023-02-15 06:55:38,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8201b09c78dcdd78cf4aad35dd3691a7: 2023-02-15 06:55:38,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,370 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=8201b09c78dcdd78cf4aad35dd3691a7, regionState=CLOSED 2023-02-15 06:55:38,371 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444138370"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444138370"}]},"ts":"1676444138370"} 2023-02-15 06:55:38,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-15 06:55:38,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 8201b09c78dcdd78cf4aad35dd3691a7, server=jenkins-hbase4.apache.org,35089,1676444032716 in 195 msec 2023-02-15 06:55:38,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-15 06:55:38,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8201b09c78dcdd78cf4aad35dd3691a7, UNASSIGN in 198 msec 2023-02-15 06:55:38,385 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=8201b09c78dcdd78cf4aad35dd3691a7, threads=14 2023-02-15 06:55:38,385 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/044f180860354ff78bfadaba17ae29bd for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,385 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/1157c02ab7f54d1d9c18db9c4956205d for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,385 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/47ea04940ff244cab7562c7530b8c4e2 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,385 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/902cafaccf25480b9f207bae187f7f78 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,386 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/a986a6cad34842d394f79f0e7a863a3a for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,388 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/aa18d11c467941f3aed635e42a1661e9 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,388 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/ec96646855974ba79554f324f09a1cec for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,388 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/061351af8a2742ccac1b54dfda902d95 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,388 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/16ba9ee4eb9842c98cafbc2548b293e8 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,388 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/21f58da3123847dfb5a4e3a99a5665b0 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,392 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/3894842d396d4e2f8f4ef8f823e90f5d for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,392 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/6850ccdb2da741909957ff3395c87635 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,392 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/b5c3b9df2d04404fb1f1a66997515797 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,392 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/efea034835904f319f8e790b58bfabb5 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,397 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/902cafaccf25480b9f207bae187f7f78, top=false 2023-02-15 06:55:38,399 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/6850ccdb2da741909957ff3395c87635, top=false 2023-02-15 06:55:38,406 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/ec96646855974ba79554f324f09a1cec, top=false 2023-02-15 06:55:38,406 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/1157c02ab7f54d1d9c18db9c4956205d, top=false 2023-02-15 06:55:38,406 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/21f58da3123847dfb5a4e3a99a5665b0, top=false 2023-02-15 06:55:38,407 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/044f180860354ff78bfadaba17ae29bd, top=false 2023-02-15 06:55:38,407 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/3894842d396d4e2f8f4ef8f823e90f5d, top=false 2023-02-15 06:55:38,409 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/aa18d11c467941f3aed635e42a1661e9, top=false 2023-02-15 06:55:38,409 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/47ea04940ff244cab7562c7530b8c4e2, top=false 2023-02-15 06:55:38,410 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/16ba9ee4eb9842c98cafbc2548b293e8, top=false 2023-02-15 06:55:38,412 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/061351af8a2742ccac1b54dfda902d95, top=false 2023-02-15 06:55:38,413 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/a986a6cad34842d394f79f0e7a863a3a, top=false 2023-02-15 06:55:38,415 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/efea034835904f319f8e790b58bfabb5, top=false 2023-02-15 06:55:38,416 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/b5c3b9df2d04404fb1f1a66997515797, top=false 2023-02-15 06:55:38,424 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-044f180860354ff78bfadaba17ae29bd for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,426 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/044f180860354ff78bfadaba17ae29bd for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,432 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-6850ccdb2da741909957ff3395c87635 for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,436 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/6850ccdb2da741909957ff3395c87635 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,446 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-b5c3b9df2d04404fb1f1a66997515797 for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,447 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-21f58da3123847dfb5a4e3a99a5665b0 for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,447 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-aa18d11c467941f3aed635e42a1661e9 for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,447 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-902cafaccf25480b9f207bae187f7f78 for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,447 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-ec96646855974ba79554f324f09a1cec for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,447 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-a986a6cad34842d394f79f0e7a863a3a for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,448 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-efea034835904f319f8e790b58bfabb5 for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,450 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-16ba9ee4eb9842c98cafbc2548b293e8 for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,450 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-47ea04940ff244cab7562c7530b8c4e2 for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,450 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-1157c02ab7f54d1d9c18db9c4956205d for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,451 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-3894842d396d4e2f8f4ef8f823e90f5d for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,454 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/a986a6cad34842d394f79f0e7a863a3a for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,456 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/b5c3b9df2d04404fb1f1a66997515797 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,456 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/21f58da3123847dfb5a4e3a99a5665b0 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,457 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/47ea04940ff244cab7562c7530b8c4e2 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,458 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/.splits/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-061351af8a2742ccac1b54dfda902d95 for child: d44023a1ed274fb3f64074c6a39d6b0c, parent: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,458 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/ec96646855974ba79554f324f09a1cec for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,459 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/1157c02ab7f54d1d9c18db9c4956205d for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,459 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/16ba9ee4eb9842c98cafbc2548b293e8 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,459 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/aa18d11c467941f3aed635e42a1661e9 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,460 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/902cafaccf25480b9f207bae187f7f78 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,460 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/efea034835904f319f8e790b58bfabb5 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,461 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/3894842d396d4e2f8f4ef8f823e90f5d for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,462 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/061351af8a2742ccac1b54dfda902d95 for region: 8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:38,462 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 8201b09c78dcdd78cf4aad35dd3691a7 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-15 06:55:38,489 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:55:38,491 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:55:38,493 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676444138492"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676444138492"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676444138492"}]},"ts":"1676444138492"} 2023-02-15 06:55:38,493 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676444138492"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444138492"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444138492"}]},"ts":"1676444138492"} 2023-02-15 06:55:38,493 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676444138492"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444138492"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444138492"}]},"ts":"1676444138492"} 2023-02-15 06:55:38,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d44023a1ed274fb3f64074c6a39d6b0c, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5b9f033095f82d0609448626e024d0d, ASSIGN}] 2023-02-15 06:55:38,496 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5b9f033095f82d0609448626e024d0d, ASSIGN 2023-02-15 06:55:38,496 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5b9f033095f82d0609448626e024d0d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,35089,1676444032716; forceNewPlan=false, retain=false 2023-02-15 06:55:38,497 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d44023a1ed274fb3f64074c6a39d6b0c, ASSIGN 2023-02-15 06:55:38,497 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d44023a1ed274fb3f64074c6a39d6b0c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,35089,1676444032716; forceNewPlan=false, retain=false 2023-02-15 06:55:38,647 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-15 06:55:38,648 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=d44023a1ed274fb3f64074c6a39d6b0c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,649 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676444138648"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444138648"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444138648"}]},"ts":"1676444138648"} 2023-02-15 06:55:38,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure d44023a1ed274fb3f64074c6a39d6b0c, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:55:38,652 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=b5b9f033095f82d0609448626e024d0d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,652 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676444138652"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444138652"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444138652"}]},"ts":"1676444138652"} 2023-02-15 06:55:38,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure b5b9f033095f82d0609448626e024d0d, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:55:38,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c. 2023-02-15 06:55:38,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d44023a1ed274fb3f64074c6a39d6b0c, NAME => 'testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.', STARTKEY => '', ENDKEY => '71'} 2023-02-15 06:55:38,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:55:38,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:38,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:38,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:38,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:38,806 INFO [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:38,806 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1 2023-02-15 06:55:38,806 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1 2023-02-15 06:55:38,807 INFO [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d44023a1ed274fb3f64074c6a39d6b0c columnFamilyName cf1 2023-02-15 06:55:38,812 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-061351af8a2742ccac1b54dfda902d95 2023-02-15 06:55:38,815 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-16ba9ee4eb9842c98cafbc2548b293e8 2023-02-15 06:55:38,817 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-21f58da3123847dfb5a4e3a99a5665b0 2023-02-15 06:55:38,820 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-3894842d396d4e2f8f4ef8f823e90f5d 2023-02-15 06:55:38,822 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-6850ccdb2da741909957ff3395c87635 2023-02-15 06:55:38,824 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-b5c3b9df2d04404fb1f1a66997515797 2023-02-15 06:55:38,827 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-efea034835904f319f8e790b58bfabb5 2023-02-15 06:55:38,827 INFO [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(310): Store=d44023a1ed274fb3f64074c6a39d6b0c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:38,827 INFO [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:38,828 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2 2023-02-15 06:55:38,828 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2 2023-02-15 06:55:38,828 INFO [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d44023a1ed274fb3f64074c6a39d6b0c columnFamilyName cf2 2023-02-15 06:55:38,832 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-044f180860354ff78bfadaba17ae29bd 2023-02-15 06:55:38,834 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-1157c02ab7f54d1d9c18db9c4956205d 2023-02-15 06:55:38,836 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-47ea04940ff244cab7562c7530b8c4e2 2023-02-15 06:55:38,839 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-902cafaccf25480b9f207bae187f7f78 2023-02-15 06:55:38,841 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-a986a6cad34842d394f79f0e7a863a3a 2023-02-15 06:55:38,844 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-aa18d11c467941f3aed635e42a1661e9 2023-02-15 06:55:38,846 DEBUG [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-ec96646855974ba79554f324f09a1cec 2023-02-15 06:55:38,847 INFO [StoreOpener-d44023a1ed274fb3f64074c6a39d6b0c-1] regionserver.HStore(310): Store=d44023a1ed274fb3f64074c6a39d6b0c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:38,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:38,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:38,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:55:38,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:38,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d44023a1ed274fb3f64074c6a39d6b0c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72222041, jitterRate=0.07619227468967438}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:55:38,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d44023a1ed274fb3f64074c6a39d6b0c: 2023-02-15 06:55:38,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c., pid=133, masterSystemTime=1676444138801 2023-02-15 06:55:38,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:38,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:55:38,853 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:55:38,853 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:55:38,853 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-15 06:55:38,853 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-15 06:55:38,853 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HStore(1912): d44023a1ed274fb3f64074c6a39d6b0c/cf1 is initiating minor compaction (all files) 2023-02-15 06:55:38,854 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HStore(1912): d44023a1ed274fb3f64074c6a39d6b0c/cf2 is initiating minor compaction (all files) 2023-02-15 06:55:38,854 INFO [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d44023a1ed274fb3f64074c6a39d6b0c/cf1 in testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c. 2023-02-15 06:55:38,854 INFO [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HRegion(2258): Starting compaction of d44023a1ed274fb3f64074c6a39d6b0c/cf2 in testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c. 2023-02-15 06:55:38,854 INFO [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-efea034835904f319f8e790b58bfabb5, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-16ba9ee4eb9842c98cafbc2548b293e8, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-061351af8a2742ccac1b54dfda902d95, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-b5c3b9df2d04404fb1f1a66997515797, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-21f58da3123847dfb5a4e3a99a5665b0, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-3894842d396d4e2f8f4ef8f823e90f5d, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-6850ccdb2da741909957ff3395c87635] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/.tmp, totalSize=34.9 K 2023-02-15 06:55:38,854 INFO [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-902cafaccf25480b9f207bae187f7f78, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-1157c02ab7f54d1d9c18db9c4956205d, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-a986a6cad34842d394f79f0e7a863a3a, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-aa18d11c467941f3aed635e42a1661e9, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-47ea04940ff244cab7562c7530b8c4e2, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-ec96646855974ba79554f324f09a1cec, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-044f180860354ff78bfadaba17ae29bd] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/.tmp, totalSize=34.9 K 2023-02-15 06:55:38,854 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-efea034835904f319f8e790b58bfabb5, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444137102 2023-02-15 06:55:38,854 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-902cafaccf25480b9f207bae187f7f78, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444137102 2023-02-15 06:55:38,855 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-16ba9ee4eb9842c98cafbc2548b293e8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444137286 2023-02-15 06:55:38,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c. 2023-02-15 06:55:38,855 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-1157c02ab7f54d1d9c18db9c4956205d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444137286 2023-02-15 06:55:38,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c. 2023-02-15 06:55:38,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d. 2023-02-15 06:55:38,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b5b9f033095f82d0609448626e024d0d, NAME => 'testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d.', STARTKEY => '71', ENDKEY => ''} 2023-02-15 06:55:38,855 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=d44023a1ed274fb3f64074c6a39d6b0c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,855 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-a986a6cad34842d394f79f0e7a863a3a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444137456 2023-02-15 06:55:38,855 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-061351af8a2742ccac1b54dfda902d95, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444137456 2023-02-15 06:55:38,855 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676444138855"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444138855"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444138855"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444138855"}]},"ts":"1676444138855"} 2023-02-15 06:55:38,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:55:38,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:38,856 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-aa18d11c467941f3aed635e42a1661e9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444137655 2023-02-15 06:55:38,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,856 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-b5c3b9df2d04404fb1f1a66997515797, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444137655 2023-02-15 06:55:38,856 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-47ea04940ff244cab7562c7530b8c4e2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444137828 2023-02-15 06:55:38,856 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-ec96646855974ba79554f324f09a1cec, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444137994 2023-02-15 06:55:38,857 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-21f58da3123847dfb5a4e3a99a5665b0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444137828 2023-02-15 06:55:38,857 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-044f180860354ff78bfadaba17ae29bd, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444138165 2023-02-15 06:55:38,857 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-3894842d396d4e2f8f4ef8f823e90f5d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444137994 2023-02-15 06:55:38,857 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-6850ccdb2da741909957ff3395c87635, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444138165 2023-02-15 06:55:38,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-02-15 06:55:38,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure d44023a1ed274fb3f64074c6a39d6b0c, server=jenkins-hbase4.apache.org,35089,1676444032716 in 207 msec 2023-02-15 06:55:38,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d44023a1ed274fb3f64074c6a39d6b0c, ASSIGN in 363 msec 2023-02-15 06:55:38,861 INFO [StoreOpener-b5b9f033095f82d0609448626e024d0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,863 DEBUG [StoreOpener-b5b9f033095f82d0609448626e024d0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/cf1 2023-02-15 06:55:38,863 DEBUG [StoreOpener-b5b9f033095f82d0609448626e024d0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/cf1 2023-02-15 06:55:38,863 INFO [StoreOpener-b5b9f033095f82d0609448626e024d0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5b9f033095f82d0609448626e024d0d columnFamilyName cf1 2023-02-15 06:55:38,864 INFO [StoreOpener-b5b9f033095f82d0609448626e024d0d-1] regionserver.HStore(310): Store=b5b9f033095f82d0609448626e024d0d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:38,864 INFO [StoreOpener-b5b9f033095f82d0609448626e024d0d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,866 DEBUG [StoreOpener-b5b9f033095f82d0609448626e024d0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/cf2 2023-02-15 06:55:38,866 DEBUG [StoreOpener-b5b9f033095f82d0609448626e024d0d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/cf2 2023-02-15 06:55:38,866 INFO [StoreOpener-b5b9f033095f82d0609448626e024d0d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b5b9f033095f82d0609448626e024d0d columnFamilyName cf2 2023-02-15 06:55:38,867 INFO [StoreOpener-b5b9f033095f82d0609448626e024d0d-1] regionserver.HStore(310): Store=b5b9f033095f82d0609448626e024d0d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:38,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:55:38,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:38,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b5b9f033095f82d0609448626e024d0d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62721913, jitterRate=-0.06537066400051117}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:55:38,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b5b9f033095f82d0609448626e024d0d: 2023-02-15 06:55:38,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d., pid=134, masterSystemTime=1676444138801 2023-02-15 06:55:38,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d. 2023-02-15 06:55:38,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d. 2023-02-15 06:55:38,875 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=b5b9f033095f82d0609448626e024d0d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,876 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676444138875"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444138875"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444138875"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444138875"}]},"ts":"1676444138875"} 2023-02-15 06:55:38,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-02-15 06:55:38,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure b5b9f033095f82d0609448626e024d0d, server=jenkins-hbase4.apache.org,35089,1676444032716 in 224 msec 2023-02-15 06:55:38,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-15 06:55:38,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5b9f033095f82d0609448626e024d0d, ASSIGN in 383 msec 2023-02-15 06:55:38,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=8201b09c78dcdd78cf4aad35dd3691a7, daughterA=d44023a1ed274fb3f64074c6a39d6b0c, daughterB=b5b9f033095f82d0609448626e024d0d in 709 msec 2023-02-15 06:55:38,905 INFO [RS:0;jenkins-hbase4:35089-longCompactions-0] throttle.PressureAwareThroughputController(145): d44023a1ed274fb3f64074c6a39d6b0c#cf2#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:55:38,905 INFO [RS:0;jenkins-hbase4:35089-shortCompactions-0] throttle.PressureAwareThroughputController(145): d44023a1ed274fb3f64074c6a39d6b0c#cf1#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:55:38,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35089] ipc.CallRunner(144): callId: 991 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:56944 deadline: 1676444198923, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7. is not online on jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:38,927 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/.tmp/cf1/057ae0b010ca408dac347dd5607c28c6 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/057ae0b010ca408dac347dd5607c28c6 2023-02-15 06:55:38,928 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/.tmp/cf2/e9d6feb17c864e7994b9c6c03f28d695 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/e9d6feb17c864e7994b9c6c03f28d695 2023-02-15 06:55:38,931 INFO [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d44023a1ed274fb3f64074c6a39d6b0c/cf1 of d44023a1ed274fb3f64074c6a39d6b0c into 057ae0b010ca408dac347dd5607c28c6(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:55:38,931 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d44023a1ed274fb3f64074c6a39d6b0c: 2023-02-15 06:55:38,931 INFO [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c., storeName=d44023a1ed274fb3f64074c6a39d6b0c/cf1, priority=9, startTime=1676444138852; duration=0sec 2023-02-15 06:55:38,932 DEBUG [RS:0;jenkins-hbase4:35089-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:38,932 INFO [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d44023a1ed274fb3f64074c6a39d6b0c/cf2 of d44023a1ed274fb3f64074c6a39d6b0c into e9d6feb17c864e7994b9c6c03f28d695(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:55:38,932 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for d44023a1ed274fb3f64074c6a39d6b0c: 2023-02-15 06:55:38,932 INFO [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c., storeName=d44023a1ed274fb3f64074c6a39d6b0c/cf2, priority=9, startTime=1676444138853; duration=0sec 2023-02-15 06:55:38,932 DEBUG [RS:0;jenkins-hbase4:35089-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:39,033 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-15 06:55:39,036 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:55:39,038 INFO [Listener at localhost/37669] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-15 06:55:39,038 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-15 06:55:39,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-15 06:55:39,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-15 06:55:39,041 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444139041"}]},"ts":"1676444139041"} 2023-02-15 06:55:39,042 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-15 06:55:39,043 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-15 06:55:39,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d44023a1ed274fb3f64074c6a39d6b0c, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5b9f033095f82d0609448626e024d0d, UNASSIGN}] 2023-02-15 06:55:39,044 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5b9f033095f82d0609448626e024d0d, UNASSIGN 2023-02-15 06:55:39,045 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=b5b9f033095f82d0609448626e024d0d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:39,045 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676444139045"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444139045"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444139045"}]},"ts":"1676444139045"} 2023-02-15 06:55:39,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure b5b9f033095f82d0609448626e024d0d, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:55:39,047 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d44023a1ed274fb3f64074c6a39d6b0c, UNASSIGN 2023-02-15 06:55:39,048 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=d44023a1ed274fb3f64074c6a39d6b0c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:39,048 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676444139048"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444139048"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444139048"}]},"ts":"1676444139048"} 2023-02-15 06:55:39,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure d44023a1ed274fb3f64074c6a39d6b0c, server=jenkins-hbase4.apache.org,35089,1676444032716}] 2023-02-15 06:55:39,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-15 06:55:39,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:39,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d44023a1ed274fb3f64074c6a39d6b0c, disabling compactions & flushes 2023-02-15 06:55:39,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c. 2023-02-15 06:55:39,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c. 2023-02-15 06:55:39,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c. after waiting 0 ms 2023-02-15 06:55:39,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c. 2023-02-15 06:55:39,198 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-efea034835904f319f8e790b58bfabb5, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-16ba9ee4eb9842c98cafbc2548b293e8, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-061351af8a2742ccac1b54dfda902d95, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-b5c3b9df2d04404fb1f1a66997515797, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-21f58da3123847dfb5a4e3a99a5665b0, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-3894842d396d4e2f8f4ef8f823e90f5d, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-6850ccdb2da741909957ff3395c87635] to archive 2023-02-15 06:55:39,199 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:55:39,201 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-efea034835904f319f8e790b58bfabb5 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-efea034835904f319f8e790b58bfabb5 2023-02-15 06:55:39,202 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-16ba9ee4eb9842c98cafbc2548b293e8 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-16ba9ee4eb9842c98cafbc2548b293e8 2023-02-15 06:55:39,203 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-061351af8a2742ccac1b54dfda902d95 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-061351af8a2742ccac1b54dfda902d95 2023-02-15 06:55:39,204 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-b5c3b9df2d04404fb1f1a66997515797 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-b5c3b9df2d04404fb1f1a66997515797 2023-02-15 06:55:39,205 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-21f58da3123847dfb5a4e3a99a5665b0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-21f58da3123847dfb5a4e3a99a5665b0 2023-02-15 06:55:39,207 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-3894842d396d4e2f8f4ef8f823e90f5d to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-3894842d396d4e2f8f4ef8f823e90f5d 2023-02-15 06:55:39,208 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-6850ccdb2da741909957ff3395c87635 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-6850ccdb2da741909957ff3395c87635 2023-02-15 06:55:39,210 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-902cafaccf25480b9f207bae187f7f78, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-1157c02ab7f54d1d9c18db9c4956205d, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-a986a6cad34842d394f79f0e7a863a3a, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-aa18d11c467941f3aed635e42a1661e9, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-47ea04940ff244cab7562c7530b8c4e2, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-ec96646855974ba79554f324f09a1cec, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-044f180860354ff78bfadaba17ae29bd] to archive 2023-02-15 06:55:39,210 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:55:39,212 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-902cafaccf25480b9f207bae187f7f78 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-902cafaccf25480b9f207bae187f7f78 2023-02-15 06:55:39,213 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-1157c02ab7f54d1d9c18db9c4956205d to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-1157c02ab7f54d1d9c18db9c4956205d 2023-02-15 06:55:39,214 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-a986a6cad34842d394f79f0e7a863a3a to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-a986a6cad34842d394f79f0e7a863a3a 2023-02-15 06:55:39,215 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-aa18d11c467941f3aed635e42a1661e9 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-aa18d11c467941f3aed635e42a1661e9 2023-02-15 06:55:39,216 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-47ea04940ff244cab7562c7530b8c4e2 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-47ea04940ff244cab7562c7530b8c4e2 2023-02-15 06:55:39,217 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-ec96646855974ba79554f324f09a1cec to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-ec96646855974ba79554f324f09a1cec 2023-02-15 06:55:39,218 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-044f180860354ff78bfadaba17ae29bd to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/testSplitTableRegionEmptyDaughter=8201b09c78dcdd78cf4aad35dd3691a7-044f180860354ff78bfadaba17ae29bd 2023-02-15 06:55:39,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-15 06:55:39,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:39,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c. 2023-02-15 06:55:39,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d44023a1ed274fb3f64074c6a39d6b0c: 2023-02-15 06:55:39,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:39,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:39,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b5b9f033095f82d0609448626e024d0d, disabling compactions & flushes 2023-02-15 06:55:39,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d. 2023-02-15 06:55:39,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d. 2023-02-15 06:55:39,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d. after waiting 0 ms 2023-02-15 06:55:39,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d. 2023-02-15 06:55:39,224 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=d44023a1ed274fb3f64074c6a39d6b0c, regionState=CLOSED 2023-02-15 06:55:39,224 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676444139224"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444139224"}]},"ts":"1676444139224"} 2023-02-15 06:55:39,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-15 06:55:39,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure d44023a1ed274fb3f64074c6a39d6b0c, server=jenkins-hbase4.apache.org,35089,1676444032716 in 177 msec 2023-02-15 06:55:39,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-15 06:55:39,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:39,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d. 2023-02-15 06:55:39,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b5b9f033095f82d0609448626e024d0d: 2023-02-15 06:55:39,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d44023a1ed274fb3f64074c6a39d6b0c, UNASSIGN in 183 msec 2023-02-15 06:55:39,229 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:39,229 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=b5b9f033095f82d0609448626e024d0d, regionState=CLOSED 2023-02-15 06:55:39,229 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676444139229"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444139229"}]},"ts":"1676444139229"} 2023-02-15 06:55:39,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-15 06:55:39,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure b5b9f033095f82d0609448626e024d0d, server=jenkins-hbase4.apache.org,35089,1676444032716 in 185 msec 2023-02-15 06:55:39,234 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-15 06:55:39,234 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b5b9f033095f82d0609448626e024d0d, UNASSIGN in 189 msec 2023-02-15 06:55:39,235 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444139234"}]},"ts":"1676444139234"} 2023-02-15 06:55:39,235 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-15 06:55:39,237 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-15 06:55:39,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 199 msec 2023-02-15 06:55:39,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-15 06:55:39,343 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-15 06:55:39,343 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-15 06:55:39,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-15 06:55:39,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-15 06:55:39,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-15 06:55:39,347 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-15 06:55:39,351 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:39,351 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:39,351 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:39,353 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/recovered.edits] 2023-02-15 06:55:39,353 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/recovered.edits] 2023-02-15 06:55:39,353 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/recovered.edits] 2023-02-15 06:55:39,356 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/057ae0b010ca408dac347dd5607c28c6 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf1/057ae0b010ca408dac347dd5607c28c6 2023-02-15 06:55:39,356 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/061351af8a2742ccac1b54dfda902d95 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/061351af8a2742ccac1b54dfda902d95 2023-02-15 06:55:39,358 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/16ba9ee4eb9842c98cafbc2548b293e8 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/16ba9ee4eb9842c98cafbc2548b293e8 2023-02-15 06:55:39,358 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/recovered.edits/88.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d/recovered.edits/88.seqid 2023-02-15 06:55:39,358 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/e9d6feb17c864e7994b9c6c03f28d695 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/cf2/e9d6feb17c864e7994b9c6c03f28d695 2023-02-15 06:55:39,359 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/b5b9f033095f82d0609448626e024d0d 2023-02-15 06:55:39,359 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/21f58da3123847dfb5a4e3a99a5665b0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/21f58da3123847dfb5a4e3a99a5665b0 2023-02-15 06:55:39,360 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/3894842d396d4e2f8f4ef8f823e90f5d to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/3894842d396d4e2f8f4ef8f823e90f5d 2023-02-15 06:55:39,361 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/recovered.edits/90.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c/recovered.edits/90.seqid 2023-02-15 06:55:39,361 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/6850ccdb2da741909957ff3395c87635 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/6850ccdb2da741909957ff3395c87635 2023-02-15 06:55:39,362 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/d44023a1ed274fb3f64074c6a39d6b0c 2023-02-15 06:55:39,363 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/b5c3b9df2d04404fb1f1a66997515797 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/b5c3b9df2d04404fb1f1a66997515797 2023-02-15 06:55:39,364 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/efea034835904f319f8e790b58bfabb5 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf1/efea034835904f319f8e790b58bfabb5 2023-02-15 06:55:39,366 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/044f180860354ff78bfadaba17ae29bd to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/044f180860354ff78bfadaba17ae29bd 2023-02-15 06:55:39,367 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/1157c02ab7f54d1d9c18db9c4956205d to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/1157c02ab7f54d1d9c18db9c4956205d 2023-02-15 06:55:39,368 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/47ea04940ff244cab7562c7530b8c4e2 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/47ea04940ff244cab7562c7530b8c4e2 2023-02-15 06:55:39,369 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/902cafaccf25480b9f207bae187f7f78 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/902cafaccf25480b9f207bae187f7f78 2023-02-15 06:55:39,370 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/a986a6cad34842d394f79f0e7a863a3a to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/a986a6cad34842d394f79f0e7a863a3a 2023-02-15 06:55:39,370 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/aa18d11c467941f3aed635e42a1661e9 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/aa18d11c467941f3aed635e42a1661e9 2023-02-15 06:55:39,371 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/ec96646855974ba79554f324f09a1cec to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/cf2/ec96646855974ba79554f324f09a1cec 2023-02-15 06:55:39,373 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/recovered.edits/85.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7/recovered.edits/85.seqid 2023-02-15 06:55:39,374 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegionEmptyDaughter/8201b09c78dcdd78cf4aad35dd3691a7 2023-02-15 06:55:39,374 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-15 06:55:39,376 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-15 06:55:39,377 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-15 06:55:39,379 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-15 06:55:39,380 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-15 06:55:39,380 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-15 06:55:39,380 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444139380"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:39,380 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444139380"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:39,380 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444139380"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:39,382 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-15 06:55:39,382 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8201b09c78dcdd78cf4aad35dd3691a7, NAME => 'testSplitTableRegionEmptyDaughter,,1676444136099.8201b09c78dcdd78cf4aad35dd3691a7.', STARTKEY => '', ENDKEY => ''}, {ENCODED => d44023a1ed274fb3f64074c6a39d6b0c, NAME => 'testSplitTableRegionEmptyDaughter,,1676444138171.d44023a1ed274fb3f64074c6a39d6b0c.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => b5b9f033095f82d0609448626e024d0d, NAME => 'testSplitTableRegionEmptyDaughter,71,1676444138171.b5b9f033095f82d0609448626e024d0d.', STARTKEY => '71', ENDKEY => ''}] 2023-02-15 06:55:39,382 ERROR [BP-145338253-172.31.14.131-1676444028112 heartbeating to localhost/127.0.0.1:41175] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-145338253-172.31.14.131-1676444028112 (Datanode Uuid 36321d6c-46ca-442e-92ed-6bcb61ed715a) service to localhost/127.0.0.1:41175 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-15 06:55:39,382 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-15 06:55:39,382 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676444139382"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:39,383 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-15 06:55:39,385 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-15 06:55:39,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 41 msec 2023-02-15 06:55:39,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-15 06:55:39,448 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-15 06:55:39,461 INFO [Listener at localhost/37669] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376 (was 376), OpenFileDescriptor=844 (was 844), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=143 (was 147), ProcessCount=168 (was 168), AvailableMemoryMB=3191 (was 3197) 2023-02-15 06:55:39,476 INFO [Listener at localhost/37669] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376, OpenFileDescriptor=844, MaxFileDescriptor=60000, SystemLoadAverage=143, ProcessCount=168, AvailableMemoryMB=3191 2023-02-15 06:55:39,476 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:55:39,481 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-15 06:55:39,481 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-15 06:55:39,482 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-15 06:55:39,483 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-15 06:55:39,484 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-15 06:55:39,486 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:39,486 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254 empty. 2023-02-15 06:55:39,486 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:39,487 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-15 06:55:39,499 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-15 06:55:39,500 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 623c848d846ec478643dfeab2ed3b254, NAME => 'testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp 2023-02-15 06:55:39,506 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:39,506 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 623c848d846ec478643dfeab2ed3b254, disabling compactions & flushes 2023-02-15 06:55:39,506 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:39,506 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:39,506 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. after waiting 0 ms 2023-02-15 06:55:39,506 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:39,506 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:39,506 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 623c848d846ec478643dfeab2ed3b254: 2023-02-15 06:55:39,508 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-15 06:55:39,509 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444139509"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444139509"}]},"ts":"1676444139509"} 2023-02-15 06:55:39,510 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-15 06:55:39,510 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-15 06:55:39,510 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444139510"}]},"ts":"1676444139510"} 2023-02-15 06:55:39,511 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-15 06:55:39,513 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-15 06:55:39,514 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-15 06:55:39,514 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-15 06:55:39,514 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-15 06:55:39,514 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-15 06:55:39,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=623c848d846ec478643dfeab2ed3b254, ASSIGN}] 2023-02-15 06:55:39,515 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=623c848d846ec478643dfeab2ed3b254, ASSIGN 2023-02-15 06:55:39,516 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=623c848d846ec478643dfeab2ed3b254, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:55:39,666 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-15 06:55:39,667 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=623c848d846ec478643dfeab2ed3b254, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:39,667 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444139667"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444139667"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444139667"}]},"ts":"1676444139667"} 2023-02-15 06:55:39,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 623c848d846ec478643dfeab2ed3b254, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:55:39,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:39,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 623c848d846ec478643dfeab2ed3b254, NAME => 'testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.', STARTKEY => '', ENDKEY => ''} 2023-02-15 06:55:39,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:55:39,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:39,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:39,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:39,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:39,827 INFO [StoreOpener-623c848d846ec478643dfeab2ed3b254-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:39,828 DEBUG [StoreOpener-623c848d846ec478643dfeab2ed3b254-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1 2023-02-15 06:55:39,828 DEBUG [StoreOpener-623c848d846ec478643dfeab2ed3b254-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1 2023-02-15 06:55:39,829 INFO [StoreOpener-623c848d846ec478643dfeab2ed3b254-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 623c848d846ec478643dfeab2ed3b254 columnFamilyName cf1 2023-02-15 06:55:39,829 INFO [StoreOpener-623c848d846ec478643dfeab2ed3b254-1] regionserver.HStore(310): Store=623c848d846ec478643dfeab2ed3b254/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:39,830 INFO [StoreOpener-623c848d846ec478643dfeab2ed3b254-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:39,830 DEBUG [StoreOpener-623c848d846ec478643dfeab2ed3b254-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2 2023-02-15 06:55:39,830 DEBUG [StoreOpener-623c848d846ec478643dfeab2ed3b254-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2 2023-02-15 06:55:39,831 INFO [StoreOpener-623c848d846ec478643dfeab2ed3b254-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 623c848d846ec478643dfeab2ed3b254 columnFamilyName cf2 2023-02-15 06:55:39,831 INFO [StoreOpener-623c848d846ec478643dfeab2ed3b254-1] regionserver.HStore(310): Store=623c848d846ec478643dfeab2ed3b254/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:39,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:39,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:39,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:55:39,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:39,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-15 06:55:39,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 623c848d846ec478643dfeab2ed3b254; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68334570, jitterRate=0.018264442682266235}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:55:39,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 623c848d846ec478643dfeab2ed3b254: 2023-02-15 06:55:39,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254., pid=143, masterSystemTime=1676444139820 2023-02-15 06:55:39,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:39,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:39,838 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=623c848d846ec478643dfeab2ed3b254, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:39,838 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444139838"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444139838"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444139838"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444139838"}]},"ts":"1676444139838"} 2023-02-15 06:55:39,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-15 06:55:39,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 623c848d846ec478643dfeab2ed3b254, server=jenkins-hbase4.apache.org,41759,1676444032818 in 171 msec 2023-02-15 06:55:39,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-15 06:55:39,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=623c848d846ec478643dfeab2ed3b254, ASSIGN in 326 msec 2023-02-15 06:55:39,842 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-15 06:55:39,842 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444139842"}]},"ts":"1676444139842"} 2023-02-15 06:55:39,843 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-15 06:55:39,845 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-15 06:55:39,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 363 msec 2023-02-15 06:55:39,989 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:39,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-15 06:55:39,990 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-15 06:55:39,990 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:39,990 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-15 06:55:39,990 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-15 06:55:39,991 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:39,991 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:39,993 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:39,993 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:39,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:39,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:39,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:39,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:39,993 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:39,993 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:39,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:39,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:39,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:39,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:39,993 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:39,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:39,993 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:39,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:39,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:39,994 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:39,994 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:39,994 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:39,994 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:39,994 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:39,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:39,995 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:39,995 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:39,995 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:39,995 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:39,995 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:39,995 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:39,995 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:39,995 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:39,995 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:39,995 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:39,995 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:39,995 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. started... 2023-02-15 06:55:39,996 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 623c848d846ec478643dfeab2ed3b254 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-15 06:55:39,996 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:39,997 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:39,997 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:39,997 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:39,997 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:39,997 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,004 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9 2023-02-15 06:55:40,019 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/910772dbef7d43aa89f04a7773f7757c 2023-02-15 06:55:40,023 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9 2023-02-15 06:55:40,026 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:55:40,026 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/910772dbef7d43aa89f04a7773f7757c as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c 2023-02-15 06:55:40,029 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:55:40,029 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 623c848d846ec478643dfeab2ed3b254 in 33ms, sequenceid=6, compaction requested=false 2023-02-15 06:55:40,030 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-15 06:55:40,030 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 623c848d846ec478643dfeab2ed3b254: 2023-02-15 06:55:40,030 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,030 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:40,030 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:40,030 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,030 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,030 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,032 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,032 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,032 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,032 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-15 06:55:40,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@786bcb51[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:40,034 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-15 06:55:40,034 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,036 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,036 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,036 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:40,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:40,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,036 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:40,037 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,037 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,037 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,037 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,038 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,038 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,038 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,047 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,047 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,047 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-15 06:55:40,050 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-15 06:55:40,050 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:40,050 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:40,050 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-15 06:55:40,050 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:40,051 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,051 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,051 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,051 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,051 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,052 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,052 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,052 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,052 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,053 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,054 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,055 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,055 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,057 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,057 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,057 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,057 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,057 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,057 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-15 06:55:40,058 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:40,057 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,057 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,057 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,057 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,058 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:40,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,058 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:40,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,058 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,058 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,058 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,059 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,059 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,059 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,059 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,059 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,059 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,158 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:40,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:40,169 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:40,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:40,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-15 06:55:40,170 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-15 06:55:40,170 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,170 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-15 06:55:40,170 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-15 06:55:40,170 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,171 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,172 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,172 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,172 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,172 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,173 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,173 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,174 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,174 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:40,174 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,174 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,174 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. started... 2023-02-15 06:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,175 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 623c848d846ec478643dfeab2ed3b254 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,176 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,177 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,177 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,177 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,177 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,177 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,186 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/34a15426ba84441a8f6855050c14b323 2023-02-15 06:55:40,197 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/47ec41c4514943e2999a45c76f3bb4b0 2023-02-15 06:55:40,201 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/34a15426ba84441a8f6855050c14b323 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323 2023-02-15 06:55:40,204 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:55:40,204 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/47ec41c4514943e2999a45c76f3bb4b0 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0 2023-02-15 06:55:40,208 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0, entries=10, sequenceid=19, filesize=5.0 K 2023-02-15 06:55:40,209 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 623c848d846ec478643dfeab2ed3b254 in 34ms, sequenceid=19, compaction requested=false 2023-02-15 06:55:40,209 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 623c848d846ec478643dfeab2ed3b254: 2023-02-15 06:55:40,209 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,209 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:40,209 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:40,209 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,209 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,209 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,211 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,211 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,211 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,211 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-15 06:55:40,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f4153e0[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:40,213 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-15 06:55:40,213 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,215 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,215 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,215 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,215 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,215 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,216 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,215 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,216 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,216 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:40,216 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:40,215 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:40,215 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,216 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,216 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,218 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,218 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,218 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,218 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,218 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:40,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-15 06:55:40,221 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-15 06:55:40,221 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:40,221 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:40,221 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-15 06:55:40,221 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:40,222 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,222 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,222 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,222 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,222 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,222 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,222 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,223 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,223 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,223 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,225 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,225 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,226 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,230 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,230 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-15 06:55:40,230 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:40,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,230 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,231 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:40,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,231 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,231 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,231 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,231 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,231 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,231 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,331 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:40,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:40,341 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:40,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:40,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-15 06:55:40,343 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-15 06:55:40,343 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,343 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-15 06:55:40,343 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-15 06:55:40,343 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,343 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,344 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,344 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,344 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,344 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,345 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,345 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:40,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,346 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,346 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,346 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,346 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,346 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,346 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,346 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,346 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:40,346 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,346 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,346 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. started... 2023-02-15 06:55:40,346 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,346 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,346 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:40,346 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,347 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 623c848d846ec478643dfeab2ed3b254 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:40,346 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,350 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,350 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,351 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,351 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,351 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,351 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,356 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f 2023-02-15 06:55:40,366 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/eff41bd4a1214d6a8891ea05ee0d8b19 2023-02-15 06:55:40,369 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f 2023-02-15 06:55:40,372 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:55:40,373 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/eff41bd4a1214d6a8891ea05ee0d8b19 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19 2023-02-15 06:55:40,376 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19, entries=10, sequenceid=32, filesize=5.0 K 2023-02-15 06:55:40,376 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 623c848d846ec478643dfeab2ed3b254 in 30ms, sequenceid=32, compaction requested=true 2023-02-15 06:55:40,377 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 623c848d846ec478643dfeab2ed3b254: 2023-02-15 06:55:40,377 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,377 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:40,377 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:40,377 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,377 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,377 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,378 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-15 06:55:40,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f4233a6[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:40,380 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-15 06:55:40,380 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,381 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,381 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,381 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,382 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,382 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,382 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:40,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,382 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,382 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,382 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,382 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:40,382 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:40,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,382 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,382 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,382 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:40,385 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,385 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,385 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,386 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,386 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,386 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,386 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,386 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,386 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,386 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:40,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-15 06:55:40,389 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-15 06:55:40,389 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:40,389 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:40,389 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-15 06:55:40,389 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:40,391 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,391 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,391 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,392 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,392 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,393 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,394 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,395 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,395 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,398 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-15 06:55:40,398 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,398 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:40,399 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:40,398 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,399 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,399 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,399 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,399 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,399 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,399 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,499 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:40,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:40,510 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:40,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:40,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-15 06:55:40,511 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-15 06:55:40,511 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,511 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-15 06:55:40,511 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-15 06:55:40,512 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,512 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,514 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,514 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,514 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,514 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,515 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,515 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,516 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,516 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,516 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,516 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,516 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,516 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,516 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,516 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,516 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,516 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,517 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,517 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,517 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,517 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,517 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,517 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:40,517 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:40,517 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,517 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. started... 2023-02-15 06:55:40,517 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 623c848d846ec478643dfeab2ed3b254 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:40,517 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,518 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,518 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,518 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,518 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,518 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,529 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4 2023-02-15 06:55:40,545 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/0dd72235ed0c4a738390d56cab651e04 2023-02-15 06:55:40,548 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4 2023-02-15 06:55:40,551 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:55:40,552 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/0dd72235ed0c4a738390d56cab651e04 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04 2023-02-15 06:55:40,555 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04, entries=10, sequenceid=45, filesize=5.0 K 2023-02-15 06:55:40,556 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 623c848d846ec478643dfeab2ed3b254 in 39ms, sequenceid=45, compaction requested=true 2023-02-15 06:55:40,556 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 623c848d846ec478643dfeab2ed3b254: 2023-02-15 06:55:40,556 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,556 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:40,556 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:40,556 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,556 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,557 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,558 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,558 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,558 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,558 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-15 06:55:40,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c888084[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:40,560 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-15 06:55:40,560 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,562 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,562 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,562 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,563 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,563 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:40,563 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,563 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,563 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,563 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,563 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,563 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:40,563 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,563 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:40,563 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:40,564 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,564 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,564 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,564 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,564 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,564 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,565 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,565 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,565 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,565 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:40,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-15 06:55:40,568 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-15 06:55:40,568 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:40,568 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:40,568 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-15 06:55:40,568 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:40,569 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,569 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,569 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,570 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,569 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,569 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,570 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,570 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,571 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,571 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,573 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,573 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,573 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,579 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,579 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,579 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,579 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,579 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,579 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,580 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-15 06:55:40,580 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:40,580 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,580 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:40,581 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:40,681 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:40,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:40,691 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:40,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:40,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-15 06:55:40,693 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-15 06:55:40,693 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,693 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-15 06:55:40,693 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-15 06:55:40,693 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,693 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,694 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,694 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,695 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,695 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:40,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,696 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,696 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,696 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,696 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,696 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,696 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,696 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,696 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,696 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,696 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,696 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,697 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,697 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,697 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,697 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:40,697 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:40,697 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,697 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. started... 2023-02-15 06:55:40,697 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 623c848d846ec478643dfeab2ed3b254 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:40,698 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,699 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,699 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,699 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,699 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,699 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,708 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/8b057c9c35114b89ac01eb90233ca0ec 2023-02-15 06:55:40,718 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/d43c1674b1a840c18b11c6a830c34c7e 2023-02-15 06:55:40,721 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/8b057c9c35114b89ac01eb90233ca0ec as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec 2023-02-15 06:55:40,724 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:55:40,725 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/d43c1674b1a840c18b11c6a830c34c7e as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e 2023-02-15 06:55:40,728 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-15 06:55:40,728 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 623c848d846ec478643dfeab2ed3b254 in 31ms, sequenceid=58, compaction requested=true 2023-02-15 06:55:40,728 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 623c848d846ec478643dfeab2ed3b254: 2023-02-15 06:55:40,728 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,728 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:40,728 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:40,728 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,729 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,729 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,730 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,730 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,730 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,730 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-15 06:55:40,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a1ff107[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:40,732 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-15 06:55:40,732 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,733 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,733 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,733 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,733 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,733 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:40,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,733 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,733 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,733 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,733 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:40,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,733 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,733 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,733 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,734 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:40,733 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:40,734 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,735 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,735 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,736 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,736 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,736 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,736 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,736 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,736 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,736 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:40,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-15 06:55:40,739 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-15 06:55:40,739 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:40,739 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:40,739 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-15 06:55:40,739 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:40,740 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,741 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,741 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,742 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,742 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,744 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,744 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,744 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,746 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,746 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,746 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,746 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,746 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,746 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,747 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-15 06:55:40,747 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,747 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:40,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,747 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,848 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:40,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:40,859 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-15 06:55:40,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-15 06:55:40,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-15 06:55:40,860 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-15 06:55:40,861 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,861 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-15 06:55:40,861 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-15 06:55:40,861 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,861 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,862 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,862 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,862 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,862 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,862 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,862 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-15 06:55:40,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-15 06:55:40,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,35089,1676444032716' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,865 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,865 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-15 06:55:40,864 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-15 06:55:40,865 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,865 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. started... 2023-02-15 06:55:40,865 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40475,1676444032771' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,865 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 623c848d846ec478643dfeab2ed3b254 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-15 06:55:40,868 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,869 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,869 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,869 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,869 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,869 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,875 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/eafbb3451fee427982ad345503ed63f8 2023-02-15 06:55:40,885 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/5f17675fa2b44d0ab896e6d681a45062 2023-02-15 06:55:40,889 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/eafbb3451fee427982ad345503ed63f8 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8 2023-02-15 06:55:40,892 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:55:40,892 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/5f17675fa2b44d0ab896e6d681a45062 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062 2023-02-15 06:55:40,895 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062, entries=10, sequenceid=71, filesize=5.0 K 2023-02-15 06:55:40,896 INFO [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 623c848d846ec478643dfeab2ed3b254 in 31ms, sequenceid=71, compaction requested=true 2023-02-15 06:55:40,896 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 623c848d846ec478643dfeab2ed3b254: 2023-02-15 06:55:40,896 DEBUG [rs(jenkins-hbase4.apache.org,41759,1676444032818)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:40,896 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-15 06:55:40,896 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-15 06:55:40,896 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,896 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-15 06:55:40,896 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-15 06:55:40,898 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-15 06:55:40,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41759,1676444032818' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-15 06:55:40,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f446bf7[Count = 0] remaining members to acquire global barrier 2023-02-15 06:55:40,900 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-15 06:55:40,900 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,901 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,901 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,901 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,901 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-15 06:55:40,901 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,901 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,901 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,901 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40475,1676444032771' in zk 2023-02-15 06:55:40,901 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,901 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,901 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-15 06:55:40,901 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-15 06:55:40,901 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41759,1676444032818' in zk 2023-02-15 06:55:40,901 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,35089,1676444032716' in zk 2023-02-15 06:55:40,910 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,910 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,910 DEBUG [member: 'jenkins-hbase4.apache.org,40475,1676444032771' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,910 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,910 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-15 06:55:40,910 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,910 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,910 DEBUG [member: 'jenkins-hbase4.apache.org,35089,1676444032716' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,910 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,910 DEBUG [member: 'jenkins-hbase4.apache.org,41759,1676444032818' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-15 06:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,41759,1676444032818': 2023-02-15 06:55:40,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41759,1676444032818' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-15 06:55:40,914 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-15 06:55:40,914 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-15 06:55:40,914 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-15 06:55:40,914 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-15 06:55:40,914 INFO [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-15 06:55:40,916 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,916 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,916 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-15 06:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-15 06:55:40,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,916 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-15 06:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,917 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,917 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-15 06:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:40,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,918 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-15 06:55:40,919 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-15 06:55:40,919 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,922 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,922 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,922 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,922 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,922 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,922 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-15 06:55:40,922 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-15 06:55:40,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,923 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,923 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-15 06:55:40,923 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-15 06:55:40,923 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-15 06:55:40,923 DEBUG [(jenkins-hbase4.apache.org,46669,1676444031550)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-15 06:55:40,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-15 06:55:40,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,923 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-15 06:55:40,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-15 06:55:40,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-15 06:55:40,923 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,923 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-15 06:55:40,924 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-15 06:55:40,924 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:40,924 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,924 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:40,924 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:40,924 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-15 06:55:40,924 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-15 06:55:41,024 DEBUG [Listener at localhost/37669] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-15 06:55:41,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-15 06:55:41,032 DEBUG [Listener at localhost/37669] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:41,033 DEBUG [Listener at localhost/37669] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=623c848d846ec478643dfeab2ed3b254, daughterA=514910fec780058e7875babe4947655e, daughterB=2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:41,034 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=623c848d846ec478643dfeab2ed3b254, daughterA=514910fec780058e7875babe4947655e, daughterB=2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:41,034 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=623c848d846ec478643dfeab2ed3b254, daughterA=514910fec780058e7875babe4947655e, daughterB=2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:41,034 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=623c848d846ec478643dfeab2ed3b254, daughterA=514910fec780058e7875babe4947655e, daughterB=2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:41,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=623c848d846ec478643dfeab2ed3b254, UNASSIGN}] 2023-02-15 06:55:41,036 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=623c848d846ec478643dfeab2ed3b254, UNASSIGN 2023-02-15 06:55:41,037 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=623c848d846ec478643dfeab2ed3b254, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:41,037 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444141037"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444141037"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444141037"}]},"ts":"1676444141037"} 2023-02-15 06:55:41,038 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 623c848d846ec478643dfeab2ed3b254, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:55:41,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 623c848d846ec478643dfeab2ed3b254, disabling compactions & flushes 2023-02-15 06:55:41,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:41,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:41,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. after waiting 0 ms 2023-02-15 06:55:41,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:41,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 623c848d846ec478643dfeab2ed3b254 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-15 06:55:41,200 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/a476d8dfd785429b83e90597c4548518 2023-02-15 06:55:41,215 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/96e361182d084495a5507d3c8d394694 2023-02-15 06:55:41,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf1/a476d8dfd785429b83e90597c4548518 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518 2023-02-15 06:55:41,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:55:41,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/.tmp/cf2/96e361182d084495a5507d3c8d394694 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694 2023-02-15 06:55:41,227 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694, entries=8, sequenceid=82, filesize=5.0 K 2023-02-15 06:55:41,227 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 623c848d846ec478643dfeab2ed3b254 in 37ms, sequenceid=82, compaction requested=true 2023-02-15 06:55:41,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-15 06:55:41,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:41,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. 2023-02-15 06:55:41,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 623c848d846ec478643dfeab2ed3b254: 2023-02-15 06:55:41,234 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,234 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=623c848d846ec478643dfeab2ed3b254, regionState=CLOSED 2023-02-15 06:55:41,234 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444141234"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444141234"}]},"ts":"1676444141234"} 2023-02-15 06:55:41,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-15 06:55:41,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 623c848d846ec478643dfeab2ed3b254, server=jenkins-hbase4.apache.org,41759,1676444032818 in 197 msec 2023-02-15 06:55:41,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-15 06:55:41,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=623c848d846ec478643dfeab2ed3b254, UNASSIGN in 200 msec 2023-02-15 06:55:41,246 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=623c848d846ec478643dfeab2ed3b254, threads=14 2023-02-15 06:55:41,247 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,247 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,247 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,247 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,247 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,247 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,247 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,248 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,248 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,248 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,248 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,252 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,256 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,256 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,371 ERROR [BP-145338253-172.31.14.131-1676444028112 heartbeating to localhost/127.0.0.1:41175] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-145338253-172.31.14.131-1676444028112 (Datanode Uuid 7246ed63-ed04-41ac-9719-168705d617e6) service to localhost/127.0.0.1:41175 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-15 06:55:41,443 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,444 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,456 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,462 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,470 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,470 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,477 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,477 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,480 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,480 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,488 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,490 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,491 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,831 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323 for region: 623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:41,831 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 623c848d846ec478643dfeab2ed3b254 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-15 06:55:42,269 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:55:42,271 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-15 06:55:42,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676444142272"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676444142272"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676444142272"}]},"ts":"1676444142272"} 2023-02-15 06:55:42,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444142272"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444142272"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444142272"}]},"ts":"1676444142272"} 2023-02-15 06:55:42,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444142272"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444142272"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444142272"}]},"ts":"1676444142272"} 2023-02-15 06:55:42,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=514910fec780058e7875babe4947655e, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=2fbc685ddeb5768c76f876b308b0fb06, ASSIGN}] 2023-02-15 06:55:42,276 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=2fbc685ddeb5768c76f876b308b0fb06, ASSIGN 2023-02-15 06:55:42,276 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=2fbc685ddeb5768c76f876b308b0fb06, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:55:42,277 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=514910fec780058e7875babe4947655e, ASSIGN 2023-02-15 06:55:42,277 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=514910fec780058e7875babe4947655e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41759,1676444032818; forceNewPlan=false, retain=false 2023-02-15 06:55:42,427 INFO [jenkins-hbase4:46669] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-15 06:55:42,429 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=514910fec780058e7875babe4947655e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:42,429 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444142429"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444142429"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444142429"}]},"ts":"1676444142429"} 2023-02-15 06:55:42,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 514910fec780058e7875babe4947655e, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:55:42,432 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=2fbc685ddeb5768c76f876b308b0fb06, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:42,432 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444142432"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444142432"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444142432"}]},"ts":"1676444142432"} 2023-02-15 06:55:42,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 2fbc685ddeb5768c76f876b308b0fb06, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:55:42,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:42,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 514910fec780058e7875babe4947655e, NAME => 'testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.', STARTKEY => '', ENDKEY => '41'} 2023-02-15 06:55:42,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:55:42,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 514910fec780058e7875babe4947655e 2023-02-15 06:55:42,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:42,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 514910fec780058e7875babe4947655e 2023-02-15 06:55:42,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 514910fec780058e7875babe4947655e 2023-02-15 06:55:42,586 INFO [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 514910fec780058e7875babe4947655e 2023-02-15 06:55:42,587 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1 2023-02-15 06:55:42,587 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1 2023-02-15 06:55:42,587 INFO [StoreOpener-514910fec780058e7875babe4947655e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 514910fec780058e7875babe4947655e columnFamilyName cf1 2023-02-15 06:55:42,600 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323-bottom 2023-02-15 06:55:42,603 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec-bottom 2023-02-15 06:55:42,607 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4-bottom 2023-02-15 06:55:42,611 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518-bottom 2023-02-15 06:55:42,614 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f-bottom 2023-02-15 06:55:42,618 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9-bottom 2023-02-15 06:55:42,622 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8-bottom 2023-02-15 06:55:42,622 INFO [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(310): Store=514910fec780058e7875babe4947655e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:42,622 INFO [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 514910fec780058e7875babe4947655e 2023-02-15 06:55:42,623 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2 2023-02-15 06:55:42,623 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2 2023-02-15 06:55:42,623 INFO [StoreOpener-514910fec780058e7875babe4947655e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 514910fec780058e7875babe4947655e columnFamilyName cf2 2023-02-15 06:55:42,635 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04-bottom 2023-02-15 06:55:42,639 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0-bottom 2023-02-15 06:55:42,642 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062-bottom 2023-02-15 06:55:42,645 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c-bottom 2023-02-15 06:55:42,649 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694-bottom 2023-02-15 06:55:42,654 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e-bottom 2023-02-15 06:55:42,657 DEBUG [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19-bottom 2023-02-15 06:55:42,658 INFO [StoreOpener-514910fec780058e7875babe4947655e-1] regionserver.HStore(310): Store=514910fec780058e7875babe4947655e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:42,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e 2023-02-15 06:55:42,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e 2023-02-15 06:55:42,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:55:42,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 514910fec780058e7875babe4947655e 2023-02-15 06:55:42,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 514910fec780058e7875babe4947655e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69300509, jitterRate=0.03265805542469025}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:55:42,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 514910fec780058e7875babe4947655e: 2023-02-15 06:55:42,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e., pid=149, masterSystemTime=1676444142582 2023-02-15 06:55:42,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:42,664 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:55:42,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:42,664 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:42,664 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:55:42,664 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1912): 514910fec780058e7875babe4947655e/cf1 is initiating minor compaction (all files) 2023-02-15 06:55:42,664 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:42,664 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 514910fec780058e7875babe4947655e/cf1 in testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:42,664 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1912): 514910fec780058e7875babe4947655e/cf2 is initiating minor compaction (all files) 2023-02-15 06:55:42,664 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 514910fec780058e7875babe4947655e/cf2 in testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:42,665 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518-bottom] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/.tmp, totalSize=34.9 K 2023-02-15 06:55:42,665 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694-bottom] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/.tmp, totalSize=34.9 K 2023-02-15 06:55:42,665 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444139985 2023-02-15 06:55:42,665 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676444139985 2023-02-15 06:55:42,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:42,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:42,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:42,666 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444140160 2023-02-15 06:55:42,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2fbc685ddeb5768c76f876b308b0fb06, NAME => 'testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.', STARTKEY => '41', ENDKEY => ''} 2023-02-15 06:55:42,666 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676444140160 2023-02-15 06:55:42,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-15 06:55:42,666 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444140333 2023-02-15 06:55:42,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:42,666 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676444140333 2023-02-15 06:55:42,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-15 06:55:42,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:42,667 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444140500 2023-02-15 06:55:42,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:42,667 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=514910fec780058e7875babe4947655e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:42,667 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444142667"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444142667"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444142667"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444142667"}]},"ts":"1676444142667"} 2023-02-15 06:55:42,667 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444140682 2023-02-15 06:55:42,667 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676444140500 2023-02-15 06:55:42,668 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444140849 2023-02-15 06:55:42,668 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676444140682 2023-02-15 06:55:42,668 INFO [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:42,668 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444141026 2023-02-15 06:55:42,668 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676444140849 2023-02-15 06:55:42,669 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1 2023-02-15 06:55:42,669 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1 2023-02-15 06:55:42,669 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676444141026 2023-02-15 06:55:42,671 INFO [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2fbc685ddeb5768c76f876b308b0fb06 columnFamilyName cf1 2023-02-15 06:55:42,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-15 06:55:42,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 514910fec780058e7875babe4947655e, server=jenkins-hbase4.apache.org,41759,1676444032818 in 238 msec 2023-02-15 06:55:42,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=514910fec780058e7875babe4947655e, ASSIGN in 396 msec 2023-02-15 06:55:42,681 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] throttle.PressureAwareThroughputController(145): 514910fec780058e7875babe4947655e#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:55:42,683 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323-top 2023-02-15 06:55:42,686 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec-top 2023-02-15 06:55:42,688 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4-top 2023-02-15 06:55:42,692 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] throttle.PressureAwareThroughputController(145): 514910fec780058e7875babe4947655e#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:55:42,697 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518-top 2023-02-15 06:55:42,706 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f-top 2023-02-15 06:55:42,713 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9-top 2023-02-15 06:55:42,718 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8-top 2023-02-15 06:55:42,718 INFO [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(310): Store=2fbc685ddeb5768c76f876b308b0fb06/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:42,718 INFO [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:42,719 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2 2023-02-15 06:55:42,719 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2 2023-02-15 06:55:42,720 INFO [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2fbc685ddeb5768c76f876b308b0fb06 columnFamilyName cf2 2023-02-15 06:55:42,726 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/.tmp/cf2/b8ef080d349f420b99720e4525277268 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/b8ef080d349f420b99720e4525277268 2023-02-15 06:55:42,728 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/.tmp/cf1/b816ecbdc77d49b68b4a5d809481c0c2 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/b816ecbdc77d49b68b4a5d809481c0c2 2023-02-15 06:55:42,731 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 514910fec780058e7875babe4947655e/cf2 of 514910fec780058e7875babe4947655e into b8ef080d349f420b99720e4525277268(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:55:42,731 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 514910fec780058e7875babe4947655e: 2023-02-15 06:55:42,731 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e., storeName=514910fec780058e7875babe4947655e/cf2, priority=9, startTime=1676444142664; duration=0sec 2023-02-15 06:55:42,731 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:42,733 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 514910fec780058e7875babe4947655e/cf1 of 514910fec780058e7875babe4947655e into b816ecbdc77d49b68b4a5d809481c0c2(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:55:42,733 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 514910fec780058e7875babe4947655e: 2023-02-15 06:55:42,733 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e., storeName=514910fec780058e7875babe4947655e/cf1, priority=9, startTime=1676444142663; duration=0sec 2023-02-15 06:55:42,733 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:42,735 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04-top 2023-02-15 06:55:42,737 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0-top 2023-02-15 06:55:42,740 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062-top 2023-02-15 06:55:42,743 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c-top 2023-02-15 06:55:42,749 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694-top 2023-02-15 06:55:42,751 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e-top 2023-02-15 06:55:42,754 DEBUG [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(539): loaded hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19-top 2023-02-15 06:55:42,755 INFO [StoreOpener-2fbc685ddeb5768c76f876b308b0fb06-1] regionserver.HStore(310): Store=2fbc685ddeb5768c76f876b308b0fb06/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-15 06:55:42,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:42,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:42,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-15 06:55:42,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:42,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2fbc685ddeb5768c76f876b308b0fb06; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69142688, jitterRate=0.030306339263916016}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-15 06:55:42,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2fbc685ddeb5768c76f876b308b0fb06: 2023-02-15 06:55:42,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06., pid=150, masterSystemTime=1676444142582 2023-02-15 06:55:42,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:42,760 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:55:42,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-15 06:55:42,760 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:42,760 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1912): 2fbc685ddeb5768c76f876b308b0fb06/cf1 is initiating minor compaction (all files) 2023-02-15 06:55:42,760 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-15 06:55:42,760 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2fbc685ddeb5768c76f876b308b0fb06/cf1 in testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:42,760 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:42,761 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1912): 2fbc685ddeb5768c76f876b308b0fb06/cf2 is initiating minor compaction (all files) 2023-02-15 06:55:42,761 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 2fbc685ddeb5768c76f876b308b0fb06/cf2 in testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:42,761 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518-top] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/.tmp, totalSize=34.9 K 2023-02-15 06:55:42,761 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694-top] into tmpdir=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/.tmp, totalSize=34.9 K 2023-02-15 06:55:42,762 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676444139985 2023-02-15 06:55:42,761 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676444139985 2023-02-15 06:55:42,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:42,762 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676444140160 2023-02-15 06:55:42,762 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676444140160 2023-02-15 06:55:42,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:42,763 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676444140333 2023-02-15 06:55:42,763 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=2fbc685ddeb5768c76f876b308b0fb06, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:42,763 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676444140333 2023-02-15 06:55:42,763 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444142763"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676444142763"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676444142763"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676444142763"}]},"ts":"1676444142763"} 2023-02-15 06:55:42,763 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676444140500 2023-02-15 06:55:42,763 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676444140500 2023-02-15 06:55:42,764 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676444140682 2023-02-15 06:55:42,764 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676444140849 2023-02-15 06:55:42,764 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting 8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676444140682 2023-02-15 06:55:42,765 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] compactions.Compactor(207): Compacting 96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676444141026 2023-02-15 06:55:42,765 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676444140849 2023-02-15 06:55:42,765 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] compactions.Compactor(207): Compacting a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676444141026 2023-02-15 06:55:42,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-15 06:55:42,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 2fbc685ddeb5768c76f876b308b0fb06, server=jenkins-hbase4.apache.org,41759,1676444032818 in 331 msec 2023-02-15 06:55:42,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-15 06:55:42,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=2fbc685ddeb5768c76f876b308b0fb06, ASSIGN in 494 msec 2023-02-15 06:55:42,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=623c848d846ec478643dfeab2ed3b254, daughterA=514910fec780058e7875babe4947655e, daughterB=2fbc685ddeb5768c76f876b308b0fb06 in 1.7390 sec 2023-02-15 06:55:42,775 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] throttle.PressureAwareThroughputController(145): 2fbc685ddeb5768c76f876b308b0fb06#cf2#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:55:42,778 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2fbc685ddeb5768c76f876b308b0fb06#cf1#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-15 06:55:42,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41759] ipc.CallRunner(144): callId: 1120 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:42146 deadline: 1676444202789, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254. is not online on jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:42,793 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/.tmp/cf2/92290c4b567b4f17989fac2d876f75e6 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/92290c4b567b4f17989fac2d876f75e6 2023-02-15 06:55:42,797 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/.tmp/cf1/a78d44ddeabc490b9cc38d0346e044f1 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/a78d44ddeabc490b9cc38d0346e044f1 2023-02-15 06:55:42,798 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2fbc685ddeb5768c76f876b308b0fb06/cf2 of 2fbc685ddeb5768c76f876b308b0fb06 into 92290c4b567b4f17989fac2d876f75e6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:55:42,798 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2fbc685ddeb5768c76f876b308b0fb06: 2023-02-15 06:55:42,798 INFO [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06., storeName=2fbc685ddeb5768c76f876b308b0fb06/cf2, priority=9, startTime=1676444142760; duration=0sec 2023-02-15 06:55:42,798 DEBUG [RS:2;jenkins-hbase4:41759-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:42,800 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2fbc685ddeb5768c76f876b308b0fb06/cf1 of 2fbc685ddeb5768c76f876b308b0fb06 into a78d44ddeabc490b9cc38d0346e044f1(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-15 06:55:42,800 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2fbc685ddeb5768c76f876b308b0fb06: 2023-02-15 06:55:42,800 INFO [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06., storeName=2fbc685ddeb5768c76f876b308b0fb06/cf1, priority=9, startTime=1676444142759; duration=0sec 2023-02-15 06:55:42,800 DEBUG [RS:2;jenkins-hbase4:41759-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-15 06:55:42,899 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-15 06:55:42,906 INFO [Listener at localhost/37669] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-15 06:55:42,907 DEBUG [Listener at localhost/37669] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-15 06:55:42,913 WARN [Listener at localhost/37669] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-15 06:55:42,915 INFO [Listener at localhost/37669] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-15 06:55:42,915 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-15 06:55:42,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-15 06:55:42,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-15 06:55:42,918 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444142918"}]},"ts":"1676444142918"} 2023-02-15 06:55:42,919 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-15 06:55:42,921 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-15 06:55:42,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=514910fec780058e7875babe4947655e, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=2fbc685ddeb5768c76f876b308b0fb06, UNASSIGN}] 2023-02-15 06:55:42,922 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=2fbc685ddeb5768c76f876b308b0fb06, UNASSIGN 2023-02-15 06:55:42,922 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=2fbc685ddeb5768c76f876b308b0fb06, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:42,923 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444142922"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444142922"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444142922"}]},"ts":"1676444142922"} 2023-02-15 06:55:42,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 2fbc685ddeb5768c76f876b308b0fb06, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:55:42,925 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=514910fec780058e7875babe4947655e, UNASSIGN 2023-02-15 06:55:42,926 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=514910fec780058e7875babe4947655e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:42,926 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444142926"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676444142926"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676444142926"}]},"ts":"1676444142926"} 2023-02-15 06:55:42,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 514910fec780058e7875babe4947655e, server=jenkins-hbase4.apache.org,41759,1676444032818}] 2023-02-15 06:55:43,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-15 06:55:43,075 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:43,075 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2fbc685ddeb5768c76f876b308b0fb06, disabling compactions & flushes 2023-02-15 06:55:43,075 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:43,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:43,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. after waiting 0 ms 2023-02-15 06:55:43,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:43,076 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518-top] to archive 2023-02-15 06:55:43,077 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:55:43,078 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,079 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,080 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,082 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,083 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,084 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,085 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,086 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062-top, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694-top] to archive 2023-02-15 06:55:43,086 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:55:43,087 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,088 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,089 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,090 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,091 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,092 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,093 DEBUG [StoreCloser-testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-15 06:55:43,096 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:43,097 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06. 2023-02-15 06:55:43,097 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2fbc685ddeb5768c76f876b308b0fb06: 2023-02-15 06:55:43,098 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:43,098 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 514910fec780058e7875babe4947655e 2023-02-15 06:55:43,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 514910fec780058e7875babe4947655e, disabling compactions & flushes 2023-02-15 06:55:43,098 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:43,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:43,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. after waiting 0 ms 2023-02-15 06:55:43,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:43,098 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=2fbc685ddeb5768c76f876b308b0fb06, regionState=CLOSED 2023-02-15 06:55:43,098 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444143098"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444143098"}]},"ts":"1676444143098"} 2023-02-15 06:55:43,099 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518-bottom] to archive 2023-02-15 06:55:43,100 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:55:43,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-15 06:55:43,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 2fbc685ddeb5768c76f876b308b0fb06, server=jenkins-hbase4.apache.org,41759,1676444032818 in 176 msec 2023-02-15 06:55:43,101 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=2fbc685ddeb5768c76f876b308b0fb06, UNASSIGN in 180 msec 2023-02-15 06:55:43,102 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/34a15426ba84441a8f6855050c14b323.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,103 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,104 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,105 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/8b057c9c35114b89ac01eb90233ca0ec.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,106 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/eafbb3451fee427982ad345503ed63f8.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,107 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/a476d8dfd785429b83e90597c4548518.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,109 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062-bottom, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254->hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694-bottom] to archive 2023-02-15 06:55:43,110 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-15 06:55:43,111 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/910772dbef7d43aa89f04a7773f7757c.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,112 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/47ec41c4514943e2999a45c76f3bb4b0.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,113 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/eff41bd4a1214d6a8891ea05ee0d8b19.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,115 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/0dd72235ed0c4a738390d56cab651e04.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,116 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/d43c1674b1a840c18b11c6a830c34c7e.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,117 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/5f17675fa2b44d0ab896e6d681a45062.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,118 DEBUG [StoreCloser-testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/96e361182d084495a5507d3c8d394694.623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-15 06:55:43,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:43,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e. 2023-02-15 06:55:43,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 514910fec780058e7875babe4947655e: 2023-02-15 06:55:43,123 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 514910fec780058e7875babe4947655e 2023-02-15 06:55:43,123 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=514910fec780058e7875babe4947655e, regionState=CLOSED 2023-02-15 06:55:43,123 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676444143123"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676444143123"}]},"ts":"1676444143123"} 2023-02-15 06:55:43,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-15 06:55:43,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 514910fec780058e7875babe4947655e, server=jenkins-hbase4.apache.org,41759,1676444032818 in 197 msec 2023-02-15 06:55:43,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-15 06:55:43,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=514910fec780058e7875babe4947655e, UNASSIGN in 204 msec 2023-02-15 06:55:43,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676444143127"}]},"ts":"1676444143127"} 2023-02-15 06:55:43,128 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-15 06:55:43,129 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-15 06:55:43,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 214 msec 2023-02-15 06:55:43,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-15 06:55:43,221 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-15 06:55:43,222 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-15 06:55:43,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-15 06:55:43,224 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-15 06:55:43,225 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-15 06:55:43,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-15 06:55:43,228 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,228 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:43,228 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e 2023-02-15 06:55:43,230 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/recovered.edits] 2023-02-15 06:55:43,230 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/recovered.edits] 2023-02-15 06:55:43,230 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2, FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/recovered.edits] 2023-02-15 06:55:43,233 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/b816ecbdc77d49b68b4a5d809481c0c2 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf1/b816ecbdc77d49b68b4a5d809481c0c2 2023-02-15 06:55:43,233 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/a78d44ddeabc490b9cc38d0346e044f1 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf1/a78d44ddeabc490b9cc38d0346e044f1 2023-02-15 06:55:43,234 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/34a15426ba84441a8f6855050c14b323 2023-02-15 06:55:43,235 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8b057c9c35114b89ac01eb90233ca0ec 2023-02-15 06:55:43,235 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/b8ef080d349f420b99720e4525277268 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/cf2/b8ef080d349f420b99720e4525277268 2023-02-15 06:55:43,236 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/92290c4b567b4f17989fac2d876f75e6 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/cf2/92290c4b567b4f17989fac2d876f75e6 2023-02-15 06:55:43,236 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/8cd90dc7e8f44ac5acf64a4d048c47d4 2023-02-15 06:55:43,238 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/a476d8dfd785429b83e90597c4548518 2023-02-15 06:55:43,238 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/recovered.edits/90.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e/recovered.edits/90.seqid 2023-02-15 06:55:43,239 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/recovered.edits/90.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06/recovered.edits/90.seqid 2023-02-15 06:55:43,239 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/514910fec780058e7875babe4947655e 2023-02-15 06:55:43,239 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/b3e2b37601bf46a3b1e1751d3d6ce33f 2023-02-15 06:55:43,239 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/2fbc685ddeb5768c76f876b308b0fb06 2023-02-15 06:55:43,240 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/e2b65c670e3e4cf5bd6738d2d49aeea9 2023-02-15 06:55:43,242 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf1/eafbb3451fee427982ad345503ed63f8 2023-02-15 06:55:43,244 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/0dd72235ed0c4a738390d56cab651e04 2023-02-15 06:55:43,245 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/47ec41c4514943e2999a45c76f3bb4b0 2023-02-15 06:55:43,246 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/5f17675fa2b44d0ab896e6d681a45062 2023-02-15 06:55:43,247 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/910772dbef7d43aa89f04a7773f7757c 2023-02-15 06:55:43,248 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/96e361182d084495a5507d3c8d394694 2023-02-15 06:55:43,249 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/d43c1674b1a840c18b11c6a830c34c7e 2023-02-15 06:55:43,250 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19 to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/cf2/eff41bd4a1214d6a8891ea05ee0d8b19 2023-02-15 06:55:43,252 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/recovered.edits/85.seqid to hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/archive/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254/recovered.edits/85.seqid 2023-02-15 06:55:43,252 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/.tmp/data/default/testSplitTableRegion/623c848d846ec478643dfeab2ed3b254 2023-02-15 06:55:43,252 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-15 06:55:43,254 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-15 06:55:43,255 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-15 06:55:43,256 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-15 06:55:43,257 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-15 06:55:43,257 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-15 06:55:43,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444143257"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:43,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444143257"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:43,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676444143257"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:43,259 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-15 06:55:43,259 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 623c848d846ec478643dfeab2ed3b254, NAME => 'testSplitTableRegion,,1676444139482.623c848d846ec478643dfeab2ed3b254.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 514910fec780058e7875babe4947655e, NAME => 'testSplitTableRegion,,1676444141032.514910fec780058e7875babe4947655e.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 2fbc685ddeb5768c76f876b308b0fb06, NAME => 'testSplitTableRegion,41,1676444141032.2fbc685ddeb5768c76f876b308b0fb06.', STARTKEY => '41', ENDKEY => ''}] 2023-02-15 06:55:43,259 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-15 06:55:43,259 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676444143259"}]},"ts":"9223372036854775807"} 2023-02-15 06:55:43,260 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-15 06:55:43,261 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-15 06:55:43,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 39 msec 2023-02-15 06:55:43,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46669] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-15 06:55:43,326 INFO [Listener at localhost/37669] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-15 06:55:43,340 INFO [Listener at localhost/37669] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=378 (was 376) - Thread LEAK? -, OpenFileDescriptor=822 (was 844), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=148 (was 143) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3182 (was 3191) 2023-02-15 06:55:43,340 INFO [Listener at localhost/37669] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-15 06:55:43,340 INFO [Listener at localhost/37669] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-15 06:55:43,341 DEBUG [Listener at localhost/37669] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x37e887de to 127.0.0.1:51507 2023-02-15 06:55:43,341 DEBUG [Listener at localhost/37669] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-15 06:55:43,342 DEBUG [Listener at localhost/37669] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-15 06:55:43,342 DEBUG [Listener at localhost/37669] util.JVMClusterUtil(257): Found active master hash=1829693025, stopped=false 2023-02-15 06:55:43,342 INFO [Listener at localhost/37669] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,46669,1676444031550 2023-02-15 06:55:43,344 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-15 06:55:43,344 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-15 06:55:43,344 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-15 06:55:43,344 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:55:43,344 INFO [Listener at localhost/37669] procedure2.ProcedureExecutor(629): Stopping 2023-02-15 06:55:43,344 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-15 06:55:43,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-15 06:55:43,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-15 06:55:43,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-15 06:55:43,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-15 06:55:43,345 DEBUG [Listener at localhost/37669] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4d68b895 to 127.0.0.1:51507 2023-02-15 06:55:43,345 DEBUG [Listener at localhost/37669] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-15 06:55:43,346 INFO [Listener at localhost/37669] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,35089,1676444032716' ***** 2023-02-15 06:55:43,346 INFO [Listener at localhost/37669] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-15 06:55:43,346 INFO [Listener at localhost/37669] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,40475,1676444032771' ***** 2023-02-15 06:55:43,346 INFO [Listener at localhost/37669] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-15 06:55:43,346 INFO [Listener at localhost/37669] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,41759,1676444032818' ***** 2023-02-15 06:55:43,346 INFO [Listener at localhost/37669] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-15 06:55:43,346 INFO [RS:0;jenkins-hbase4:35089] regionserver.HeapMemoryManager(220): Stopping 2023-02-15 06:55:43,346 INFO [RS:1;jenkins-hbase4:40475] regionserver.HeapMemoryManager(220): Stopping 2023-02-15 06:55:43,346 INFO [RS:2;jenkins-hbase4:41759] regionserver.HeapMemoryManager(220): Stopping 2023-02-15 06:55:43,346 INFO [RS:1;jenkins-hbase4:40475] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-15 06:55:43,346 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-15 06:55:43,346 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-15 06:55:43,347 INFO [RS:1;jenkins-hbase4:40475] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-15 06:55:43,347 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-15 06:55:43,347 INFO [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:43,347 DEBUG [RS:1;jenkins-hbase4:40475] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5cd38ce0 to 127.0.0.1:51507 2023-02-15 06:55:43,347 DEBUG [RS:1;jenkins-hbase4:40475] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-15 06:55:43,347 INFO [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,40475,1676444032771; all regions closed. 2023-02-15 06:55:43,347 INFO [RS:0;jenkins-hbase4:35089] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-15 06:55:43,347 INFO [RS:0;jenkins-hbase4:35089] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-15 06:55:43,347 INFO [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(3304): Received CLOSE for 03b216dbc5303ada2e264ea03896de42 2023-02-15 06:55:43,348 INFO [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:43,348 DEBUG [RS:0;jenkins-hbase4:35089] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x01e7a4d7 to 127.0.0.1:51507 2023-02-15 06:55:43,348 DEBUG [RS:0;jenkins-hbase4:35089] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-15 06:55:43,348 INFO [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-15 06:55:43,348 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1479): Online Regions={03b216dbc5303ada2e264ea03896de42=hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42.} 2023-02-15 06:55:43,348 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1505): Waiting on 03b216dbc5303ada2e264ea03896de42 2023-02-15 06:55:43,347 INFO [RS:2;jenkins-hbase4:41759] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-15 06:55:43,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 03b216dbc5303ada2e264ea03896de42, disabling compactions & flushes 2023-02-15 06:55:43,349 INFO [RS:2;jenkins-hbase4:41759] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-15 06:55:43,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:55:43,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:55:43,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. after waiting 0 ms 2023-02-15 06:55:43,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:55:43,522 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 03b216dbc5303ada2e264ea03896de42 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-15 06:55:43,521 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-15 06:55:43,521 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-15 06:55:43,522 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-15 06:55:43,522 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-15 06:55:43,349 INFO [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:43,521 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-15 06:55:43,522 DEBUG [RS:2;jenkins-hbase4:41759] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x392cd4d9 to 127.0.0.1:51507 2023-02-15 06:55:43,522 DEBUG [RS:2;jenkins-hbase4:41759] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-15 06:55:43,523 INFO [RS:2;jenkins-hbase4:41759] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-15 06:55:43,523 INFO [RS:2;jenkins-hbase4:41759] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-15 06:55:43,523 INFO [RS:2;jenkins-hbase4:41759] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-15 06:55:43,523 INFO [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-15 06:55:43,521 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-15 06:55:43,521 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-15 06:55:43,521 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-15 06:55:43,525 INFO [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-15 06:55:43,525 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-15 06:55:43,525 DEBUG [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-15 06:55:43,525 DEBUG [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-15 06:55:43,525 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-15 06:55:43,525 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-15 06:55:43,525 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-15 06:55:43,525 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-15 06:55:43,525 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-15 06:55:43,525 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-15 06:55:43,531 DEBUG [RS:1;jenkins-hbase4:40475] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/oldWALs 2023-02-15 06:55:43,531 INFO [RS:1;jenkins-hbase4:40475] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40475%2C1676444032771:(num 1676444034539) 2023-02-15 06:55:43,532 DEBUG [RS:1;jenkins-hbase4:40475] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-15 06:55:43,532 INFO [RS:1;jenkins-hbase4:40475] regionserver.LeaseManager(133): Closed leases 2023-02-15 06:55:43,532 INFO [RS:1;jenkins-hbase4:40475] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-15 06:55:43,532 INFO [RS:1;jenkins-hbase4:40475] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-15 06:55:43,532 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-15 06:55:43,532 INFO [RS:1;jenkins-hbase4:40475] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-15 06:55:43,533 INFO [RS:1;jenkins-hbase4:40475] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-15 06:55:43,534 INFO [RS:1;jenkins-hbase4:40475] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40475 2023-02-15 06:55:43,542 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:43,542 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:43,542 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-15 06:55:43,542 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-15 06:55:43,542 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40475,1676444032771 2023-02-15 06:55:43,543 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@73d89a3f rejected from java.util.concurrent.ThreadPoolExecutor@77e7109c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:43,543 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-15 06:55:43,543 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@34234daf rejected from java.util.concurrent.ThreadPoolExecutor@77e7109c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:43,542 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-15 06:55:43,544 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40475,1676444032771] 2023-02-15 06:55:43,544 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40475,1676444032771; numProcessing=1 2023-02-15 06:55:43,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42/.tmp/info/6736f961c24a4b5c8d870e8fbfd8e645 2023-02-15 06:55:43,545 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,40475,1676444032771 already deleted, retry=false 2023-02-15 06:55:43,545 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40475,1676444032771 expired; onlineServers=2 2023-02-15 06:55:43,549 DEBUG [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1505): Waiting on 03b216dbc5303ada2e264ea03896de42 2023-02-15 06:55:43,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42/.tmp/info/6736f961c24a4b5c8d870e8fbfd8e645 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42/info/6736f961c24a4b5c8d870e8fbfd8e645 2023-02-15 06:55:43,551 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/.tmp/info/1b938016af0a4dd08f35ce4844254879 2023-02-15 06:55:43,552 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42/info/6736f961c24a4b5c8d870e8fbfd8e645, entries=2, sequenceid=6, filesize=4.8 K 2023-02-15 06:55:43,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 03b216dbc5303ada2e264ea03896de42 in 31ms, sequenceid=6, compaction requested=false 2023-02-15 06:55:43,555 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1b938016af0a4dd08f35ce4844254879 2023-02-15 06:55:43,557 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/namespace/03b216dbc5303ada2e264ea03896de42/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-15 06:55:43,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:43,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:55:43,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 03b216dbc5303ada2e264ea03896de42: 2023-02-15 06:55:43,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676444034787.03b216dbc5303ada2e264ea03896de42. 2023-02-15 06:55:43,563 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/.tmp/rep_barrier/0a5d8623b4da4a6db9ceed2b01707907 2023-02-15 06:55:43,566 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a5d8623b4da4a6db9ceed2b01707907 2023-02-15 06:55:43,573 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/.tmp/table/26fe3ded9a75459ca52c502bab27690a 2023-02-15 06:55:43,576 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 26fe3ded9a75459ca52c502bab27690a 2023-02-15 06:55:43,576 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/.tmp/info/1b938016af0a4dd08f35ce4844254879 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/info/1b938016af0a4dd08f35ce4844254879 2023-02-15 06:55:43,579 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1b938016af0a4dd08f35ce4844254879 2023-02-15 06:55:43,579 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/info/1b938016af0a4dd08f35ce4844254879, entries=59, sequenceid=210, filesize=11.1 K 2023-02-15 06:55:43,580 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/.tmp/rep_barrier/0a5d8623b4da4a6db9ceed2b01707907 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/rep_barrier/0a5d8623b4da4a6db9ceed2b01707907 2023-02-15 06:55:43,583 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a5d8623b4da4a6db9ceed2b01707907 2023-02-15 06:55:43,583 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/rep_barrier/0a5d8623b4da4a6db9ceed2b01707907, entries=24, sequenceid=210, filesize=7.5 K 2023-02-15 06:55:43,583 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/.tmp/table/26fe3ded9a75459ca52c502bab27690a as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/table/26fe3ded9a75459ca52c502bab27690a 2023-02-15 06:55:43,586 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 26fe3ded9a75459ca52c502bab27690a 2023-02-15 06:55:43,586 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/table/26fe3ded9a75459ca52c502bab27690a, entries=36, sequenceid=210, filesize=8.0 K 2023-02-15 06:55:43,587 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 62ms, sequenceid=210, compaction requested=false 2023-02-15 06:55:43,590 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-15 06:55:43,591 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-15 06:55:43,591 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-15 06:55:43,592 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-15 06:55:43,592 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: 2023-02-15 06:55:43,592 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-15 06:55:43,644 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-15 06:55:43,644 INFO [RS:1;jenkins-hbase4:40475] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,40475,1676444032771; zookeeper connection closed. 2023-02-15 06:55:43,644 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4cb1a369 rejected from java.util.concurrent.ThreadPoolExecutor@77e7109c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:43,644 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:40475-0x10056c1c0920002, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-15 06:55:43,644 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7caa0e88 rejected from java.util.concurrent.ThreadPoolExecutor@77e7109c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:43,644 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@856dc6b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@856dc6b 2023-02-15 06:55:43,725 INFO [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,41759,1676444032818; all regions closed. 2023-02-15 06:55:43,730 DEBUG [RS:2;jenkins-hbase4:41759] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/oldWALs 2023-02-15 06:55:43,730 INFO [RS:2;jenkins-hbase4:41759] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41759%2C1676444032818.meta:.meta(num 1676444034516) 2023-02-15 06:55:43,735 DEBUG [RS:2;jenkins-hbase4:41759] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/oldWALs 2023-02-15 06:55:43,735 INFO [RS:2;jenkins-hbase4:41759] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41759%2C1676444032818:(num 1676444034538) 2023-02-15 06:55:43,735 DEBUG [RS:2;jenkins-hbase4:41759] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-15 06:55:43,735 INFO [RS:2;jenkins-hbase4:41759] regionserver.LeaseManager(133): Closed leases 2023-02-15 06:55:43,735 INFO [RS:2;jenkins-hbase4:41759] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-15 06:55:43,736 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-15 06:55:43,736 INFO [RS:2;jenkins-hbase4:41759] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41759 2023-02-15 06:55:43,739 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:43,739 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41759,1676444032818 2023-02-15 06:55:43,739 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-15 06:55:43,739 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3849863e rejected from java.util.concurrent.ThreadPoolExecutor@3c7605ef[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:43,740 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41759,1676444032818] 2023-02-15 06:55:43,740 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41759,1676444032818; numProcessing=2 2023-02-15 06:55:43,741 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41759,1676444032818 already deleted, retry=false 2023-02-15 06:55:43,741 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41759,1676444032818 expired; onlineServers=1 2023-02-15 06:55:43,749 INFO [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,35089,1676444032716; all regions closed. 2023-02-15 06:55:43,754 DEBUG [RS:0;jenkins-hbase4:35089] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/oldWALs 2023-02-15 06:55:43,754 INFO [RS:0;jenkins-hbase4:35089] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C35089%2C1676444032716:(num 1676444034537) 2023-02-15 06:55:43,754 DEBUG [RS:0;jenkins-hbase4:35089] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-15 06:55:43,754 INFO [RS:0;jenkins-hbase4:35089] regionserver.LeaseManager(133): Closed leases 2023-02-15 06:55:43,754 INFO [RS:0;jenkins-hbase4:35089] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-15 06:55:43,754 INFO [RS:0;jenkins-hbase4:35089] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-15 06:55:43,754 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-15 06:55:43,754 INFO [RS:0;jenkins-hbase4:35089] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-15 06:55:43,754 INFO [RS:0;jenkins-hbase4:35089] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-15 06:55:43,755 INFO [RS:0;jenkins-hbase4:35089] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35089 2023-02-15 06:55:43,756 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,35089,1676444032716 2023-02-15 06:55:43,756 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-15 06:55:43,756 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5e7634b6 rejected from java.util.concurrent.ThreadPoolExecutor@5a49bba0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:43,758 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,35089,1676444032716] 2023-02-15 06:55:43,758 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,35089,1676444032716; numProcessing=3 2023-02-15 06:55:43,759 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,35089,1676444032716 already deleted, retry=false 2023-02-15 06:55:43,760 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,35089,1676444032716 expired; onlineServers=0 2023-02-15 06:55:43,760 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,46669,1676444031550' ***** 2023-02-15 06:55:43,760 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-15 06:55:43,760 DEBUG [M:0;jenkins-hbase4:46669] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c33ec5a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-15 06:55:43,760 INFO [M:0;jenkins-hbase4:46669] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,46669,1676444031550 2023-02-15 06:55:43,760 INFO [M:0;jenkins-hbase4:46669] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,46669,1676444031550; all regions closed. 2023-02-15 06:55:43,760 DEBUG [M:0;jenkins-hbase4:46669] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-15 06:55:43,760 DEBUG [M:0;jenkins-hbase4:46669] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-15 06:55:43,760 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-15 06:55:43,761 DEBUG [M:0;jenkins-hbase4:46669] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-15 06:55:43,761 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676444033929] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676444033929,5,FailOnTimeoutGroup] 2023-02-15 06:55:43,761 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676444033930] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676444033930,5,FailOnTimeoutGroup] 2023-02-15 06:55:43,761 INFO [M:0;jenkins-hbase4:46669] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-15 06:55:43,761 INFO [M:0;jenkins-hbase4:46669] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-15 06:55:43,761 INFO [M:0;jenkins-hbase4:46669] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-15 06:55:43,761 DEBUG [M:0;jenkins-hbase4:46669] master.HMaster(1512): Stopping service threads 2023-02-15 06:55:43,761 INFO [M:0;jenkins-hbase4:46669] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-15 06:55:43,761 ERROR [M:0;jenkins-hbase4:46669] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-15 06:55:43,761 INFO [M:0;jenkins-hbase4:46669] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-15 06:55:43,761 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-15 06:55:43,761 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-15 06:55:43,761 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-15 06:55:43,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-15 06:55:43,762 DEBUG [M:0;jenkins-hbase4:46669] zookeeper.ZKUtil(398): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-15 06:55:43,762 WARN [M:0;jenkins-hbase4:46669] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-15 06:55:43,762 INFO [M:0;jenkins-hbase4:46669] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-15 06:55:43,762 INFO [M:0;jenkins-hbase4:46669] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-15 06:55:43,762 DEBUG [M:0;jenkins-hbase4:46669] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-15 06:55:43,762 INFO [M:0;jenkins-hbase4:46669] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-15 06:55:43,762 DEBUG [M:0;jenkins-hbase4:46669] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-15 06:55:43,762 DEBUG [M:0;jenkins-hbase4:46669] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-15 06:55:43,763 DEBUG [M:0;jenkins-hbase4:46669] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-15 06:55:43,763 INFO [M:0;jenkins-hbase4:46669] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.15 KB heapSize=719.04 KB 2023-02-15 06:55:43,782 INFO [M:0;jenkins-hbase4:46669] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.15 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d7da064fb1924c3d9c66eb220e43ff46 2023-02-15 06:55:43,785 INFO [M:0;jenkins-hbase4:46669] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7da064fb1924c3d9c66eb220e43ff46 2023-02-15 06:55:43,786 DEBUG [M:0;jenkins-hbase4:46669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d7da064fb1924c3d9c66eb220e43ff46 as hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d7da064fb1924c3d9c66eb220e43ff46 2023-02-15 06:55:43,790 INFO [M:0;jenkins-hbase4:46669] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7da064fb1924c3d9c66eb220e43ff46 2023-02-15 06:55:43,790 INFO [M:0;jenkins-hbase4:46669] regionserver.HStore(1080): Added hdfs://localhost:41175/user/jenkins/test-data/ac764241-8012-41e9-9b1c-ab029ef9f7c2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d7da064fb1924c3d9c66eb220e43ff46, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-15 06:55:43,791 INFO [M:0;jenkins-hbase4:46669] regionserver.HRegion(2947): Finished flush of dataSize ~608.15 KB/622742, heapSize ~719.02 KB/736280, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 28ms, sequenceid=1282, compaction requested=false 2023-02-15 06:55:43,792 INFO [M:0;jenkins-hbase4:46669] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-15 06:55:43,792 DEBUG [M:0;jenkins-hbase4:46669] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-02-15 06:55:43,804 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-15 06:55:43,804 INFO [M:0;jenkins-hbase4:46669] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-15 06:55:43,805 INFO [M:0;jenkins-hbase4:46669] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46669 2023-02-15 06:55:43,807 DEBUG [M:0;jenkins-hbase4:46669] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,46669,1676444031550 already deleted, retry=false 2023-02-15 06:55:43,840 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-15 06:55:43,840 INFO [RS:2;jenkins-hbase4:41759] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,41759,1676444032818; zookeeper connection closed. 2023-02-15 06:55:43,840 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3c4bef22 rejected from java.util.concurrent.ThreadPoolExecutor@3c7605ef[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:43,841 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:41759-0x10056c1c0920003, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-15 06:55:43,841 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4492cb79 rejected from java.util.concurrent.ThreadPoolExecutor@3c7605ef[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:43,841 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3f52da9] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3f52da9 2023-02-15 06:55:43,941 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-15 06:55:43,941 INFO [M:0;jenkins-hbase4:46669] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,46669,1676444031550; zookeeper connection closed. 2023-02-15 06:55:43,941 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@d659273 rejected from java.util.concurrent.ThreadPoolExecutor@7ed856be[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:43,941 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): master:46669-0x10056c1c0920000, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-15 06:55:43,941 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@785a9cc0 rejected from java.util.concurrent.ThreadPoolExecutor@7ed856be[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:44,023 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-15 06:55:44,041 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-15 06:55:44,041 INFO [RS:0;jenkins-hbase4:35089] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,35089,1676444032716; zookeeper connection closed. 2023-02-15 06:55:44,041 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@41225ae4 rejected from java.util.concurrent.ThreadPoolExecutor@5a49bba0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:44,041 DEBUG [Listener at localhost/37669-EventThread] zookeeper.ZKWatcher(600): regionserver:35089-0x10056c1c0920001, quorum=127.0.0.1:51507, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-15 06:55:44,041 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@10d17c25] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@10d17c25 2023-02-15 06:55:44,041 ERROR [Listener at localhost/37669-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@399d3f2a rejected from java.util.concurrent.ThreadPoolExecutor@5a49bba0[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-15 06:55:44,042 INFO [Listener at localhost/37669] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-15 06:55:44,042 WARN [Listener at localhost/37669] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-15 06:55:44,045 INFO [Listener at localhost/37669] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-15 06:55:44,074 WARN [BP-145338253-172.31.14.131-1676444028112 heartbeating to localhost/127.0.0.1:41175] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-145338253-172.31.14.131-1676444028112 (Datanode Uuid 36321d6c-46ca-442e-92ed-6bcb61ed715a) service to localhost/127.0.0.1:41175 2023-02-15 06:55:44,077 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data5/current/BP-145338253-172.31.14.131-1676444028112] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-15 06:55:44,077 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data6/current/BP-145338253-172.31.14.131-1676444028112] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-15 06:55:44,079 WARN [Listener at localhost/37669] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-15 06:55:44,081 INFO [Listener at localhost/37669] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-15 06:55:44,193 WARN [BP-145338253-172.31.14.131-1676444028112 heartbeating to localhost/127.0.0.1:41175] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-15 06:55:44,194 WARN [BP-145338253-172.31.14.131-1676444028112 heartbeating to localhost/127.0.0.1:41175] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-145338253-172.31.14.131-1676444028112 (Datanode Uuid f7dac7ef-ce9e-4233-96d7-09bfa9824e44) service to localhost/127.0.0.1:41175 2023-02-15 06:55:44,196 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data3/current/BP-145338253-172.31.14.131-1676444028112] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-15 06:55:44,196 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data4/current/BP-145338253-172.31.14.131-1676444028112] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-15 06:55:44,197 WARN [Listener at localhost/37669] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-15 06:55:44,200 INFO [Listener at localhost/37669] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-15 06:55:44,305 WARN [BP-145338253-172.31.14.131-1676444028112 heartbeating to localhost/127.0.0.1:41175] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-145338253-172.31.14.131-1676444028112 (Datanode Uuid 7246ed63-ed04-41ac-9719-168705d617e6) service to localhost/127.0.0.1:41175 2023-02-15 06:55:44,305 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data1/current/BP-145338253-172.31.14.131-1676444028112] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-15 06:55:44,306 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/0ad2d0a6-dabc-2a48-24a7-802c6912a1ef/cluster_029794ca-b72b-b54f-14d2-e5d0347f0b35/dfs/data/data2/current/BP-145338253-172.31.14.131-1676444028112] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-15 06:55:44,330 INFO [Listener at localhost/37669] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-15 06:55:44,517 INFO [Listener at localhost/37669] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-15 06:55:44,757 INFO [Listener at localhost/37669] hbase.HBaseTestingUtility(1293): Minicluster is down